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This  report  presients  a  2-year,  longitudinal 
evaluation  of  the  effects  of  a  major  training  program  on  two  State 
Educational  agencies  and  four  local  educational  agencies.  The  program 
was  designed  to  improve  the  process  of  organizational  planning  in 
educational  systems.  The  evaluation  procedure  involved  interviewing 
top  administrators  who  underwent  training  four  times  over  a  2-year 
period.  Questionnaires  were  administered  to  these  and  to  HO 
additional  key  personnel.  Additionally,  an  analysis  was  conducted  of 
documents  and  samples  that  were  produced  through  the  training  program 
and  of  the  extent  to  which  they  were  implemented.  Assessed  were  429 
measures,  including  133  measures  of  written  documents  produced 
through  the  training  program  and  ?96  produced  through 
interviews/questionnaires.  The  achievement  of  the  training  program  as 
evidenced  by  written  documents  proved  to  be  only  modestly  successful. 
An  assessment  of  the  296  intervi aw/questionnaire  measures  found  that 
in  36  cases  the  program  in  organizational  planning  had  produced 
negative  effects,  in  222  cases  it  Md  produced  no  effects,  and  38 
cases  positive  effects.  The  findings  indicated  that  as  a  result  of 
the  training  program,  top  administrators  and  key  decisionmakers  at 
both  the  state  educational  agency  level  and  at  the  local  educational 
agency  level  did  not,  in  general,  change  either  their  beliefs  about 
organizational  planning  or  their  perceptions  of  how  planning  might  be 
pursued  by  the  organization.  (Author/JF) 
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This  report  is  a  two-year  evaluation  program 
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trators. The  first  year's  evaluation  report  was  completed 
and  submitted  on  October  4,  1971  under  Project  No,  0-0793, 
Grant  No.  OEG-0-70-5073. 
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PREFACE 


Undertaking  a  report  such  as  this  almost  inevitably 
becomes  a  ccMnplicated,  frequently  dismaying  and  occa- 
sionally exciting  adventure  in  group  research*  This 
preface  outlines  the  chronology  and  the  division  of 
responsibility  for  our  study,  and  acknowledges  the  many 
contributions  that  produced  it# 

The  project  began  in  August,  1970  with  a  meeting 
between  myself  and  !Ir«  Lynn  Tamner  at  the  Federal  Execu- 
tive Institute  in  Charlottesville ,  Virginia.    Mr^  Tanner 
was  then  a  doctoral  student  in  the  Public  Administration 
Program  of  the  Ilaxwell  School,  Syracuse  University.    I  was 
about  to  join  the  political  science  faculty  at  the  same 
institution*    At  the  end  of  our  first  meeting,  an  all-day 
session,  we  emerged  with  details  of  a  new  research  design 
that   was  necessary  to  strengthen  the  AMA's  original 
proposal  and  offer  potentially  greater  validity  to  the 
research  findings  as  they  became  available,    ^e  explored 
thoroughly  all  potential  areas  of  measurement  that  might 
appropriately  be  included  in  the  evaluation,  eliminated 
many,  added  new  ones,  and  strengthened  some  measures 
that  we  agreed  ware  imoortant  but  vulnerable. 

At  the  end  of  the  day,  fir.  Tanner  and  I  believed 
we  had  an  intellectually  sound,  practically  useful 
research  methodology;,  and  set  of  research  instruments.  V^e 
could  not,  however,  be  certain  that  a  control  group  would 
be  added  to  the  study.    The  American  Management  Associa- 
tion was  very  cooperative  in  aiding  and  insuring  that  a 
comparable  State  Education  Agency  that  would  not  receive 
the  training  program  would  be  included  in  the  study. 
Top  administrators  in  the  two  experimental  states  also 
were  helpful  in  suggesting  State  Educational  Agencies 
which  they  believed  were  compareible  to  their  own  and  to  the 
other  experimental  Educational  Agency •    With  their  help, 
the  control  group  was  secured  by  summer's  end. 

The  job  of  field  research  began;    rir.  Tanner  was 
in  the  field  at  once,  gathering  the  pre-training  research 
data,  conducting  interviews  and  becoming  acquainted  with 
personnel  of  both  the  training  organization  and  the  States 
to  be  studied.    In  the  year  that  followed,  Mr«  Tanner 
spent  many  days  observing  every  phase  of  the  training 
program,  taking  detailed  notes  on  the  material  presented 
by  the  AtlA  during  its  program,  and  strengthening 
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relationships  with  personnel  of  all  participating  organi- 
zations in  order  to  gain  a  clear  sense  of  events  that 
unfolded  as  the  program  and  the  evaluative  research 
proceeded. 

Toward  the  end  of  June,  1971,  all  research  data 
had  been  collectad  and  the  tedious  job  of  processing  the 
infonnat'on  began.    The  questionnaires  were  cc>ded  and 
submitted  to  computer  analysis,  not  without  the  problems 
which  usually  occur  at  this  stage  of  a  research  effort. 
Mr.  Emanuel  Wald,  a  doctoral  student  in  Public  Administra- 
tion at  Syracuse  University,  spent  hours  struggling  with 
the  problem  of  writing  statistical  programs,  insuring 
their  procedural  integrity  and  applying  the  programs  to 
our  data. 

At  this  juncture  Mr.  Kent  Chabotar  and  M.r.  Stephen 
Montgomery,  both  doctoral  students  in  Public  Administration, 
became  directly  involved  with  the  initial  A!4A  research 
project.    Both  had  participated  in  the  original  meetings 
which  set  up  the  first  year's  evaluation  project,  but  they 
had  decided  to  concentrate  on  a  second-year  studv  of  the 
training's  on-site  impacts  rather  than  on  the  training 
program  itself.    To  this  end,  they  applied  for,  and 
received  from  the  United  States  Office  of  Education,  a 
grant  to  conduct  this  second-year  evaluation. 

Messrs.  Chabotar  and  Montgomery  began  by  helping 
to  cociplete  the  first-year  report.    Theirs  was  the 
difficult  task  of  assisting  Mr.  Tanner  in  content-coding 
the  tape-recorded,  opon-onded  interviews  he  had  conducted. 
The  research  team  deeply  appreciates  the  secretarial  holo 
which  the  At'?A  generously  provided  for  the  onerous  task  of 
transcribing  the  first  year's  interview  tapes  into 
typescript. 

The  problem  of  Messrs.  Tanner,  Chabotar,  and  IJont- 
gomery  was  to  develop  a  coding  instrument  that  was  fully 
understood  and  meaningful  to  all  three,  and  then  to  use 
this  instrument  to  code  the  interviews  in  a  reliable 
fashion.    After  a  great  deal  of  deliberation,  conflict 
and  frustration,  a  viable  instrument  was  produced  and  the 
coding  was  accomplished.    At  last  the  coded  information 
was  Iceypunchod  and  subjected  to  computer  analysis.  :'7g 
labored  mightily  here,  for  we  were  convincoil  that  this 
research  data  must  be  trec.tcd  with  maximum  rigor  and  in  a 
manner  that  was  equitable  to  all  parties. 

In  writing  the  report  itself,  I  wanted  to  nroduce 
a  document  that    explicated  as  fully  as  possible  the 
contents  of  the  Mlh  program,  how  it  was  conducted,  when  it 
was  implemented  and  with  whom,  and  how  much  validity 
could  be  attributed  to  the  research  findings  due  to  the 
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nature  of  the  underlying  research  design.    I  v/as  equally 
concerned  with  choosing  types  of  statistical  analyses , 
and  with  how  the  results  of  analysis  would  be  used  to  make 
decisions  about  the  program's  consequences  during  the  first 
year  of  svibsequent  operation  in  the  experimental  organiza- 
tions. 

In  order  to  clarify  these  concerns ,  Mr,  Tanner  and  I 
worked,  quite  literally,  night  and  day  for  45  days.  Great 
mounds  of  planning  and  program  docizments  were  provided  by 
the  AMA  to  Mr.  Tanner;  he  had  also  taken  many  detailed 
notes  as  he  observed  the  programs.    In  many  cases,  none  of 
the  documents  provided  precisely  the  information  we  needed 
about  the  underlying  assumptions  and  design  parameters  of 
the  training  program.    Consequently,  we  were  forced  to 
examine  all  of  the  material  with  a  careful,  critical  eye. 

Because  of  the  press  of  time  in  writing  the 
initial  report,  we  felt  it  was  impossible  to  produce  a 
document  authored  equally  by  both  of  us.    I  took  responsi- 
bility for  writing  the  report,  and  did  so.    I5r.  Tanner's 
assistance  throughout  this  period  was  extremely  crucial, 
for  he  spent  many  hours  sharing  his  observational  notes 
and  providing  other  important  information  which  could  have 
been  gained  only  in  the  field.    Finally,  after  about  1600 
man-hours  of  work  the  first  year's  report  was  finished 
and  forwarded  to  the  AflA  and  the  Office  of  Education  in 
October,  1971. 

The  second-year's  evaluation  effort  as  proposed  by 
!Ir.  Montgomery  and  Mr.  Chabotar  and  funded  by  the  Office 
of  Education  began  immediately  after  the  first  year 
evaluation  had  been  submitted. 

Meetings  were  arranged  among  Messrs.  Montgomery, 
Chabotar,  Tanner  and  myself  to  deepen  Messrs.  Hontgomery 
and  Chabotar' s  understanding  of  events  during  the  first 
year  of  the  evaluation  and  to  lay  groundwork  for  evaluating 
the  second  year.    Mr.  Montgomery,  Mr.  Chabotar  and  I  were 
committed  to  improving  the  research  process  during  the 
second  year  and  to  refining  the  research,  instruments  used 
during  the  first  year. 

These  meetings  produced,  to  cite  two  examples,  a 
reliability  study  of  the  research  questionnaire  and  the 
inclusion  of  a  second  Control  State.    The  latter  was 
necessary  because  we  had  reason  to  believe  that  the 
original  Control  State  would  become  involved  in  the  2i!IA 
training  program  during  the  second  year's  evaluation. 
The  questionnaire's  reliability  study  was  designed  to 
insure  that  the  instrument  we  had  chosen  would  be  reliable 
in  the  research  setting.    The  research  instrument  had  been 


ustd  many  times  and  carefully  studied  before  we  araplied  it 
to  educational  systems.    Results  of  the  reliability  study 
m  our  setting  wore  gratifying.    It  strengthened  our 
confidence  in  the  first  year's  findings  and  in  the  forth- 
coming findings  of  the  second  year. 

A  number  of  new  measures  were  added  for  the  second 
year  study;  these  measures  were  geared  to  examine  the  output 
factors  that  could  be  attributed  to  the  training  program. 
Wg  wanted  to  ask  the  question,  "How  much  difference  did  the 
training  program  make  in  the  output  of  the  organization?" 
To  our  way  of  thinking,  it  was  both  appropriate  and  essen- 
tial that  this  dimension  be  added  to  the  second- year  effort. 
By  the  end  of  the  second  year,  at  least  one  year  and  in 
some  cases  one  and  one-half  years  would  have  passed  since 
the  training  program— enough  time  to  enable  us  reasonably 
to  expect  organizational  results  from  the  training.  We 
decided  also  to  include  the  Local  Education  Agencies  in  the 
final  or  second  year  report  for  they  had  been  excluded 
from  the  first  year's  study. 

Messrs.  Chabotar  and  Montgoraery  began  almost  immedi- 
ately to  gather  the  third  and  later  the  fourth  blocks  of 
research  data.    Given  what  I,  as  project  director,  had 
learned  from  the  first  year,  Messrs.  Montgomery  and  Chabotar 
and  myself    were  able  to  work  out  a  schedule  which  would 
enable  us  to  complete  the  report  before  the  official  dead- 
line for  submission. 

The  second  year's  study  incorporates  many  features 
of  the  first  year's  report  and  rests  on  that  study. 

Much  of  the  report,  particularly  the  introductory 
chapters  and  the  chapter  on  research  design  rely  heavily 
upon  the  first  year's  report.    Frankly,  I  have  made  a 
number  of  contributions  to  the  written  text  and  must  be 
held  responsible  for  the  overall  tone  and  final  interpreta- 
tion of  the  results.    Responsibility  for  any  substantive 
errors  in  the  portrayal  of  the  AMA  program,  the  analysis 
of  the  research  design  or  the  interpretation  of  statistical 
data    must  rest  with  me.    In  addition,  new  material  has 
been  added  and  key  segments  of  the  initial  report  have 
been  eled>orated.    Since  we  were  able  to  foresee  the 
schedule  which  would  be  necessary  to  produce  a  final 
report  on  time,  this  second-year  evaluation  was  written 
almost  solely  by  Mr.  Chabotar  and  Mr.  Montgomery. 

Having  offered  these  caveats,  I  urge  the  reader  to 
recognize  that  the  time-consuming  and  frustrating  produc- 
tion of  e.  report  which  incorporates  the  firet  year  study, 
builds  and  extends  through  another  year  the  measurement 
of  the  variables  studied  originally,  and  adds  and 
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interprets  new  material  from  the  second  vGar^  is  almost 
entirely  the  consequence  of  the  painstaking  Icibors  of  ^lr. 
Chabotar  and  fir.  Montgomery. 

This  preface  would  be  woefully  incomplete  if  we 
failed  to  acknowledge  the  help  of  Dr.  Frank  Marini, 
Director  of  the  Public  Administration  Program  at  Syracuse 
University^  and  that  of  his  extraordinary  secretary, 
Mrs#  Alondra  Mariani. 

On  nvunerous  occasions  I  took  significant  tine  from 
Or.  riarini's  extremely  busy  schedule  to  disouss  problems 
of  our  research.    He  was  always  helpful,  offering  useful 
suggestions  without  presumption  and  li8t':jiing  attentively. 
He  is  also  to  be  thanked  for  his  editorial  work  on  the 
first  year's  report?  Dr.  Marini  generously  and  at  virtually 
any  time  and  place  reviewed  my  text,  often  disrupting 
both  his  personal  and  professional  life  to  do  so.  During 
the  second  year  of  the  study.  Dr.  Marini  worked  to  insure 
that  administrative  and  budgetary  questions  were  resolved 
to  the  satisfaction  of  all  parties,  and  often  and  willingly 
lent  his  ear  for  debriefing  sessions  similar  to  those  of 
the  first  year. 

Mrs.  Alondra  Mariani  is  largely  responsible  for 
keeping  the  research  effort  moving  along.    To  her  fell 
the  difficult  task  of  staying  abreast  of  the  accounting 
and  administrative  details  of  the  project.    She  wns 
unfailingly  helpful  despite  the  added  burden  thr.t  the 
research  project  laid  upon  her  already  heavy  workload, 
and  we  are  deeply  indebted  to  her. 

Finally,  it  should  be  added  that,  as  research 
director,  I  was  ma.^-velously  gratified  to  watch  three 
graduate  students  work  responsibly  and  intensely  together: 
each  made  invaluable  contributions  to  the  research.  As 
the  most  significant  indication  of  the  collaborative 
relationship  they  developed,  each  will  prepare  a  doctoral 
dissertation  that  incorporates  data  developed  during  both 
years  of  the  study.    This  is,  to  me,  a  remarkable  achieve- 
ment given  the  inherent  competitiveness  of  graduate  educa- 
tion and  the  tremendous  ego  investment  that  accompanies 
dissertation  research.    On  these  grounds  alone,  though 
many  others  could  be  added,  it  was  gratifying  to  work 
closely  with  Messrs.  Chabotar,  Montgomery,  and  Tanner. 

This  preface  set  out  to  show  the  complexity  of 
completing  a  research  effort  in  wnat  can  bo  truly 
described  as  a  temporary  social  system.    As  this  report 
is  completed,  the  members  of  this  refiearch  team  leave 
Syracuse  University  for  posts  across  the  U.S.    Mr.  Tanner 
has  accepted  an  appointment  as  assistant  professor  of 
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public  administration  at  Florida  International  University; 
Mr.  Montgomery  becomes  a  consultant  for  planning  and  manage- 
ment development  to  the  planning  division  of  one  of  the 
experimental  State  Departments  of  Education  in  our  study; 
Mr.  Chabotar  has  been  appointed  an  instructor  in  political 
science  at  Michigan  State  University,  while  I  have  joined 
the  political  science  faculty  of  Louisiana  State  University 
at  New  Orleans. 

I  close  this  preface  with  mixed  feelings,  therefore; 
it  marks  the  symbolic  end  of  a  two- year  project  which  has 
been,  from  the  start,  turbulent,  often  fun,  sonetimes 
disappointing  and  marked  by  long  hours  of  work. 

Toward  the  end,  the  load  was  lightened  by  the  ideas, 
support  and  empathy  of  my  closest  friend  and  confidante, 
Cleire  Donaher  Kirkhart,  who  was  willing  to  sacrifice  many 
weeks  of  the  first  seven  months  of  our  meorriage  to  this 
report. 


Larry  Kirkhart, 
Research  Director 
July  17,  1972 

New  Orleans,  Louisiana 
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SUMfiARY 


This  report  presents  a  two-year^  longitudinal 
evaluation  of  the  effects  of  a  major  training  program 
upon  two  State  Educational  Agencies  and  four  Local 
Educational  Agencies.    The  program  was  designed  by  the 
American  Management  Association  to  in^rove  the  process  of 
organizational  planning  in  educational  systems. 

An  elaborate  evaluation  procedure  was  developed  to 
provide  a  careful  assessment  of  the  consequences  of  the 
training  program.    Top  administrators  who  underwent  the 
major  portioM  of  the  training  effort  were  interviewed 
four  times;    once  before  training  began  and  three  times 
after  the  program  was  over.    These  interviews  were  spread 
evenly  over  a  two-year  period.    In  addirion^  questionnaires 
were  administered  to  these  same  administrators  and  to 
approximately  40  additional  key  organizational  personnel^ 
at  four  intervals  during  the  same  period. 

An  analysis  or  sairiples  of  the  planning  documents 
which  were  produced  through  the  training  program  and  of 
the  extent  to  which  they  were  implemented  was  also  made. 

The  research  design  involved  a  control  group  which 
provided  a  base  of  con5>arison  in  order  to  effectively 
determine  the  results  of  the  program. 

Both  the  State  Educational  Agencies  emd  the  Local 
Educational  Agencies  went  through  the  training  program 
and  both  levels  were  encompassed  by  the  evaluation. 

Every  effort  was  made  to  design  the  study  around 
the  goalp  the  training  organization  set  out  to  achieve. 
Tn  order  to  clarify  the  presentation  of  a  complex  set  of 
empirical  findings^  the  data  were  organized  into  three 
categories:    Causal^  Intervening  and  End-Result  variables. 

Causal  variables  were  directly  related  to  the  stated 
goals  of  the  training  program;  they  were  the  results  the 
AMA  set  out  to  accomplish.    Intervening  variables  were 
defined  as  the  modes  in  which  the  organization  functioned 
internally;  matters  such  as  decision-making /  leadership / 
and  team-management  were  considered.    Each  of  the  factors 
we  have  called  intervening  variables  were  important 
considerations  in  the  presentation  of  the  AMA  training 
effort. 
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End-Result  variables  are  related,  to  organizational 
output;  progress  toward  the  accompli shment  of  an  objective 
spelled  out  in  the  organizational  plan  is  one  example; 
another  would  be  attitudes  and  beliefs  of  organization 
members  about  progress  toward  accomplishing  their  objectives. 

In  each  oategory,  a  large  number  of  measures  were 
assessed.    This  information  is  summarized  in  the  table 
below.    The  reader  will  note  that  each  category  is  broken 
into  two  sub-headings,  assessment  of  written  documents 
and  interview /questionnaire  data.    The  text  offers  an 
explanation  of  precisely  how  each  sub- area  was  assessed. 
The  written  documents  were  judged  against  criteria  which 
are  regarded  by  professional  personnel  as  essential 
elements  in  an  effective    written  plan.    The  interview/ 
questionnaire  material  was  subjected  to  careful  statistical 
analysis. 


EVALUATION  RESULTS 


Area 


Total  No. 
of  Measures 


Results 


1.    Causal  Variables 

a.  Written 
Documents 

b.  Interview/ 
Questionnaire 
Data 


Intervening 
Variables 

a.  Interview/ 
Questionnaire 
Data 


83 


184 


94 


Minimum 


21 


Negative 


24 


Negative 


11 


Moderate 


43 


No  Effect,  Positive 


141 


NO  Effect  Positive 


66 


Maximum 


19 


19 


17 


End- Re suit 
VariablGs 

a.  Written 
Documents 

b.  Interviews/ 
Questionnaire 
Data 


Unaccept 


50 


27 


Negative 


NO  Effect 


23 

Positive 


18 


15 


429 
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A  total  of  429  measures  were  assessed/  including  133 
measures  of  written  documents  produced  whrough  the  training 
program  and  296  measures  produced  through  interviews/ques- 
tionnaires.   Evaluation  of  the  written  planning  documents 
was,  at  best,  very  difficult;  thus  every  effort  was  made 
to  be  as  generous  as  possible  in  interpreting  and  applying 
evaluative  criteria.    Bassed  on  this  procedure,  48  of  the 
measures  were  found  to  be  at  minimum  level,  66  at  a 
moderate  level  of  development  and  19  at  the  level  we  term 
acceptable  or  maximum*    The  latter  implies  that  the  organi- 
zation under  scrutiny  produced  a  written  statement  that  is 
unambiguous,  clear  and  appropriately  comprehensive; 
"moderate"  implies  that  the  written  material  was  under- 
standable but  was,  in  almost  all  cases,  incomplete •  An 
"unacceptable"  or  "minimum"  ranking  indicates  that  the 
measured  area  was  not  addressed  at  all  or  done  poorly • 

We  conclude,  therefore,  that  achievement    of  the 
training  program  as  evinced  by  written  documents  (organi- 
zational planning  materials)  was  only  modestly  successful • 

Finally,  we  assessed  a  total  of  296  interview/ques- 
tionnaire measures  and  found  that  in  36  cases  the  ATIA 
program  in  organizational  planning  had  produced  negative 
effects,  in  222  cases  it  had  produced  no  effects,  and  in 
38  cases  positive  effects* 

This  is  an  extraordinarily  poor  level  of  achievement 
by  any  standard.    In  short,  as  a  result  of  the  training 
program,  top  administrators  and  key  decisionmakers  at  both 
the  State  Educational  Agency  level  and  the  Local  Educational 
?gency  level  did  not,  in  general,  change  either  their 
beliefs  about  organizational  planning  or  tl  ^ir  perceptions 
of  how  planning  might  be  pursued  by  the  organization. 

The  text  that  follows  comprehensively  analyzes 
how  the  AMA  effort  affected  each  of  the  six  experimental 
organizations.    Briefly,  what  occurred  is  this:    On  the 
basis  of  the  before-training  measures,  one  of  the  experi- 
mental organizations  began  at  a  much  higher  level  of 
performance  than  did  the  other  five.    At  the  close  of  our 
study,  it  was  the  only  organization  to  show  even  modest 
improvement  as  a  consequence  of  the  program. 

Overall,  we  are  forced  to  conclude  that  the  AMA 
training  effort  was  unsuccessful  during  these  two  years. 
Given  the  cost  oF^e  program,  the  large  number  of  manhours 
it  abso^Eed,  angPthe  map or  investment  ot  .scarce  Human 
energy  it  required,  we  conclude  that  the  program's  rewards 
are  insuTficient  to  warrant  the  investment  of  pupTic  funds. 
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PART  I 

ADAPTING  AND  TESTING  BUSINESS  MANAGEMENT  PROGRAMS 
FOR  EDUCATIONAL  ADMINISTRATORS 


ERIC 


INTRODUCTION 


In  a  fascinating  study  of  the  political  vagaries 
and  administrative  vicissitudes  that  accompanied 
implementation  of  the  1965  Elementary  and  Secondary 
Education  hot,  Stephen  Bailey  and  Edith  Mosher  comment: 

When  ESEA  was  in  its  first  weeks 
and  months  of  implementation  •  •  • 
the  infrastructure  of  systematic 
program  evaluation  was  either  ^ 
nonexistent  or  woefully  primitive. 

Since  American  educators  have  been  preoccupied  with 
problems  of  evaluation  for  over  75  years,  producing 
hundreds  of  books,  articles,  case  studies  and  research 
reports  on  the  issue,  such  a  statement  is  unsettling*^ 

But  it  is  essentially  correct.    Much  of  what  has 
passed  for  evaluation  has  been  judged  mediocre  at  best.^ 
Individual  student  or  cohort  performance  has  been 
overemphasized  at  the  expense  of  program,  school  district, 
or  larger  concerns*    The  absence  of  an  intermediate  level 
of  analysis  often  forced  a  shotgun  marriage  between  subjec- 
tive descriptions  and  data  displays  or  computer  printouts. 


Stephen  K*  Bailey  and  Edith  llosher,  ESEA:  The 
Office  of  Education  Administers  a  Law  (Syracuse:  Syracuse 
IJniversity  Press,  1968),'  p.  102. 

^See  J*  M*  Rice,  *ThG  Futility  of  the  Spelling 
Grind,"  Forum,  XXIII  (April/June,  1897),  pp.  163-172, 
409-419;  Standards  and  Tests  for  the  rieasurement  of  the 
Efficiency  of  Schools  and  school  Systems,  15th  Yearbook 
of  the  National  Society  for  the  Study  of  Education,  Part 
2  (Chicago:    University  of  Chicago  Press,  1916);  L*  J* 
Cronbach,  •'Course  Improvement  Through  Evaluation,  *•  Teachers 
College  Record,  LXIV  (May,  1963)^  pp*  672-683. 
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See  Marvxn  Bressler  and  Melvin  Tumin,  Evaluation 
of  the  Effectiveness  of  Educational  Systems,  Vol.  1 
(Princeton,  N.J#:    Princeton  University  Press,  1969); 
Orville  P.  Poland,  "Why  Does  Public  Administration  Ignore 
Evaluation?"  Public  Administration  Review,  XXXI,  2  (March/ 
April,  1971),  p.  201. 
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yet  forged  no  logical  supporting  links  between  them. 
Judgments  of  effectiveness  were  based  on  evaluations  of 
varied  input  data  (student/teacher  ratios,  per  pupil 
expenditures,  etc.)  with  only  passing  consideration  of 
the  educational  system's  output,  of  its  actual 
accomplishments . 5 

This  last  concern  expresses  what  in  our  view  is  the 
most  significant  limitation  of  current  evaluations:  the 
failure  to  compare  precisely  promise  with  performance  so  as 
to  separate  effective  from  ineffective  programs. 

It  is  odd  with  so  much  investment 
of  hope  we  know  so  little  about  the 
precise  nature  of  the  interrela- 
tionships between  istated  aims  and 
actual  outcomes.    Apparently  we  now 
lack  both  the  intellectual  apparatus— 
i.e.  standards,  theories,  concepts, 
indicators,  tests,  and  new  data*-**and 
sufficient  will  that  will  permit  us 
to  distinguish  a  'success'  from  a 
•failure. '6 

Without  such  tools,  we  cannot  be  certain  that  the 
projects  we  sponsor  (whether  WIA  training  programs  or 
reading  programs  for  inner-city  children)  are  really  worth 
our  money.    Recent  negative  findings  about  some  aspects 
of  performance  contracting  make  it  clear  that  no  easy 
solutions  exist. 


In  Organizations  (New  York:    Wiley  &  Sons,  1958), 
March  and  Simon  discuss  the  application  of  a  means'-end 
chain  connecting  nonoperational  organizational  goals  and 
individual  task  assignments  as  an  aid  to  program  evaluation. 

^For  examples  of  this  input  oriented  approach  see: 
National  Study  of  Secondary  School  Standards,  Evaluative 
Criteria  (Washington,  D.C.,  1960);  North  Central  Associa- 
tion of  Colleges  and  Secondary  Schools,  Policies  and  Cri** 
teria  for  the  Approval  of  Secondary  Schools  (Commission 
on  Secondary  Schools,  1965-66)  as  noted  by  Henry  Dyer, 
State-Wide  Evaluation  -  What  are  the  Priorities?  (Prince- 
ton,  N.J.:    Educational  Testing  Service,  1969),  p.  8.  See 
also  Norman  Kurland,  "Developing  Indicators  of  Educational 
Perfoinnance,"  a  lecture  presented  at  the  31st  Educational 
Conference  sponsored  by  the  Educational  Record  Bureau 
(October,  1966),  for  a  critique  of  this  approach. 
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Brassier  and  Tumin,  op.  cit.,  p.  2. 
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The  evaluative  situation  has  been  improving^  however. 
The  involvement  of  experimental  states  in  the  AflA  training 
program  indicates  that  state  and  local  education  agencies 
are  moving  toward  comprehensive  assessment  of  where  they 
are^  where  they  want  to  go^  and  how  they  will  recognize  the 
goal  when  thoy  reach  it  .7  This  phenomenon  is  caused  partly 
by  /:ederal  auidelines  that  require  systematic  program 
evaluation*^    It  has  been  influenced  by  improved  evaluations 
in  the  literature. ^    But  mostly  it  emerges  from  urgent 
awareness  that  we  must  be  able  to  assess  whether  the  pro- 
grams we  administer  and  the  projects  we  fund  are  helping 
the  children  we  teach  and  reaching  the  goals  we  set. 

This  trend  can  be  demonstrated,  at  least  in  part, 
through  growing  emphasis  on  testing  in  statewide  assessment 
programs,    A  1968  Educational  Testing  Service  survey  showed 
that  74  such  testing  programs  already  existed  in  42  states, 
with  18  states  offering  2  or  more  programs.    Twenty-two  (22) 
of  the  reporting  states  were  concerned  primarily  to  provide 
schools  with  tests  for  use  in  guidance;  17  supplied  tests 
as  a  means  of  instructional  evaluation;  and  13  stressed  tie 


The  evaluative  efforts  of  several  state  educational 
agencies  are  described  in  Joan  S.  Beers,  Educational 
Quality  Assessment;    The  Ton  Goals  of  Quality  Education 
(Harrisburg:    Pennsylvania  Department  of  Education,  1970) ; 
James  Ilitchell  et  ai,.  Program  Planning  and  Evaluation 
(Des  Hoines,  Iowa:    State  Department  of  Public  Instruction, 
1971);  Keith  Crusi,  Educational  Needs  Assessment;    A  State- 
wide Design  ^f or  Texas  (Austin t    State  Education  Agency, 
1971);  California  School  Boards  Association,  Implementation 
&  CSBA  (Sacramento,  1968). 

o 

Elementary  and  Secondary  Education  Act,  Public  Law 
89-10,  89th  Congress,  HR  2362,  Sec.  205(a),   (5)  and  (6); 
see  also;    Norman  Thomas,  ^'Politics,  Administration,  and 
American  Education,"  Public  Administration  Review,  XXX,  6 
(November/December ,  1970) ,  p*  649. 
g 

For  a  nonreprcsentativo  sample  of  the  more  rrcont 
studios  see  Joint  Federal/State  Task  Force  on  Evaluation, 
Comprehensive  Evaluation  System;    Current  Status  and 
Deyelppmental  Requirements  (Washington,  P.O.;  Scientific 
Educational  Systems,  Inc*,  1970);  Jack  C*  Merwin,  '^Evalua- 
tion  Designs  and  Instruments,"  from  a  symposium  "The  World 
of  Evaluation  Needs  Reshaping"  at  the  annual  meeting  of 
AERA  (Pebruaiy,  1969);  James  W.  Guthrie,  "A  Survey  of  School 
Ef fectivenesfi  Studies,"  a  paper  presented  at  the  annual 
meeting  of  A)iRA  (March,  1970)  . 
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evaluation  of  individual  student  progress.       More  recent 
evidence  indicates  that  not  only  are  more  testing  programs 
being  administered  in  more  states  and  localities,  but 
student  achievement  is  more  heavily  stressed  as  well. 

In  program  evaluation,  as  in  public  administration 
generally,  there  is  no  "one  best  way."    The  alternative 
techniques  and  criteria  are  too  numerous  to  be  listed 
here.    As  C.  Robert  Pace  points  out:    "...  the  character- 
istics of  good  evaluation  differ  depending  on  what  is 
being  evaluated,  why,  and  by  whom.    Evaluation  cannot  be 
described  by  a  single  set  of  rules. Yet  approaches  to 
evaluation  generally  rest  upon  three  preconditions  of 
empirical  methods 

1.  Reliability:    Whatever  the  evaluative 

instruments  measure,  they 
measure  consistently. 

2.  Validity:         The  instruments  measure 

what  they  are  supposed  to 
measure. 

3.  Relevance:       Evaluation  results  are 

germane  to  experiential 
problems /  not  merely  to 
theoretical  constructs. 
Whenever  evidence  produced 
by  an  instrument  continu- 
ally fails  to  affirm  ex- 
periential observations  and 
theory  derived  from  it,  the 
evaluation  itself  may  appro- 
priately be  questioned. 13 


Educational  Testing  Service,  State  Testing  Prc-^ 
grams:    A  Survey  of  Functions,  Tests /  Materials,  and 
services  (Princeton,  N.J. ,  1968). 

^^C.  Robert  Pace,  Evaluation  Perspectives  1968  (Los 
Angeles:    Center  for  the  Study  of  Evaluation,  University 
of  California  at  Los  Angeles,  1968). 

12 

Adapted  from  Ben  K.  Gold,  "Evaluation  of  Programs," 
'  paper  presented  at  a  conference  sponsored  by  the  Compensa- 

tory Education  Project,  Coordinating  Bocurd,  Texas  College 
and  University  System,  5-6  April  1971. 

13 

Egon  6.  Cuba  provided  us  with  a  dramatic  example 
at  the  1969  AERA  meeting.    It  concerned  the  "evaluation" 
O  .  of  tho  Higher  Horizons  program  in  New  York  City.  "Test 
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Even  the  most  valid,  reliable,  relevant  evaluation 
plan  will  fail  unless  the  evaluators  carefully  administer 
it,  and  unless  the  people  being  evaluated  appreciate  the 
plan's  usefulness  and  commit  themselves  to  implementing 
it.    This  commonplace  is  rarely  realized  in  the  field. 
According  to  Robert  Randall,  there  is  "a  timeworn  and  oft- 
recurring  spectacle  of  the  frantic  but  finally  productive 
researcher-evaluator,  who  rushed  into  the  executive 
offices  with  his  data  analysis  finally  complete,  his 
report  prepared  and  in  hand,  only  to  find  that  the  execu- 
tives, several  months  previously,  had  made  the  in^ortant 
decisions  that  locked  up  the  monies  and  committed  the 
organization  1  u  the  ensuing  months  ahead. "14 

Finally,  all  parties  must  realize  that  while  more 
should  be  demanded  of  evaluation  theory  and  practice  than 
we  have  so  far  received,  we  should  not  err  by  demanding 
more  than  evaluation  can  give. 15    problems  in  controlling 
for  environmental  determinants, 16  unanticipated  interven- 
tions, respondent  bias,  political  influencesl^  etc.;  make 


data  failed  to  affirm  what  supervisors,  teachers,  and  clients 
insisted  was  true — that  the  program  was  making  a  difference 
so  great  that  it  simply  could  not  be  abandoned." 

14 

Robert  S.  Randall,  "Knowledge  7\bout  Decision  Pro- 
cesses and  Information,"  paper  presented  at  the  annual 
meeting  of  AERA  (February,  1969) . 

15 

Henry  Dyer,  "Kcw  Precise  Can  Measurement  Be?"  in 
Evaluation  and  Christian  Education  (New  York:  National 
Council  of  Churches,  I960);  Robert  E.  Stake,  "Language, 
Rationality,  and  Assessment,"  in  Walcott  Beatty  (ed.). 
Improving  Educational  Asisessment  and  an  Inventory  of 
Measures  of  Affective  Behavior  (Washington,  D.C;    As soci- 
ation  for  Supervision  and  Curriculum  Development,  1969) , 
pp.  35ff. 

^^See  Austin  D.  Swanson,  "Cost-Utility  ?»nalysis  and 
Educational  Decision-Making,"  in  Gerald  G.  Mansergh,  Systems 
Approaches  to  the  Management  of  Public  Education  (Detroit: 
Metropolitan  Detroit  Bureau  of  school  Studies,  Inc.,  1969), 
pp.  15-16;  see  also:    Paul  R.  Mott  and  Orlando  Furno,  Theory 
and  Synthesis  of  a  Sequential  Simplex:    A  Model  for  Assessing 
the  Effectiveness  of  Administrative  Policies  (New  Yorlc;  The 
institute  of  Administrative  Research,  Teachers  College, 
Columbia  University,  1960) .    Mott  and  Furno  conclude  that 
about  2/3  of  the  variance  in  educational  output  is  due  to 
the  environment,  leaving  only  1/3  of  the  variance  to  be 
explained  by  decisions  of  school  boards  and  administrators. 
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J.  R.  Schlesinger,  Systems  Analysis  and  the  Politi* 
cal  Process  (Santa  Monica,  California:    The  HAND  Corporation, 
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attempts  to  give  "money-back  guarantees"  of  any  evaluation, 
particularly  in  education,  fatuous.    On  the  other  hand,  the 
need  for  educational  evaluation  is  so  pressing  that  any 
attempt,  however  approximate,  seems  more  useful  than  none. 

The  present  report  continues  evaluation  of  the 
/ancrican  Management  Association's  pilot  project  to  apply 
to  educational  administration  modified  management  and' 
planning  techniques  dravm  from  business  and  industry. 
The  report  is  divided  into  three  parts. 

Part  I  reviews  the  M!A»3  training  program,  "Adapting 
and  Testing  Business  Management  Development  Programs  for 
Educational  Administrators,"  which  was  funded  by  the  USOE 
(Contract  #  OEG-0-70-5070)  during  the  period  7/1/70-6/30/71. 
The  First  year's  evaluation  (Yi) (1970-1971)  and  findingslS 
are  also  reviewed  and  the  focus  of  the  present  evaluation 
(Y2) (1971-1972)  will  be  presented.    This  section  intends 
to  provide  an  overview  of  the  training  project  and  the 
two-year  evaluation. 

Part  II  details  the  research  methodology  of  this 
project.    The  relationship  between  the  First  and  Second 
year's  evaluations  is  presented,  together  with  the  design, 
methodologies,  data-gathering  techniques,  and  statistical 
tools  employed.    Attention  will  also  be  given  to  questions 
of  validity  and  their  implications  for  our  summary  findings. 

Part  III  presents  the  findings  and  analysis  of  the 
evaluation  project.    Again,  the  connection  between  the 
First  and  Second  year  evaluation  projects  is  emphasized. 
As  in  the  first  year  evaluation,  most  of  the  present 
evaluation  studies  the  impact  of  training  on  the  State 
Education    Departments  (SED's).    The  training  program's 


1967),  pp.  7,  29.    See  also:    Aaron  Wildavsky,  "The 
Political  Economy  of  Efficiency:    Cost  Benefit  Analysis, 
Systems  Analysis,  and  Program  Budgeting,"  Public  Adminis- 
tration Review  (December,  1966) ,  pp.  292-3T(r 

18 

This  report  was  submitted  to  the  Office  of  Educa- 
tion with  only  Frank  Marini's  nsaae  on  the  cover  page;  it 
was,  however,  never  Dr.  Marini's  intent  to  receive  credit 
for  authorship  of  the  report.    The  cover  page  appeared  as 
it  did    simply  because  of  a  misunderstanding  of  legal 
requirements . 

The  report  was  actually  authored  by  Larry  Kirkhart, 
the  research  director,  and  Mr.  Lynn  Tanner.    Hereafter  the 
present  report  will  refer  to  the  Kirkhart  and  Tanner 
authorship.    Larry  Kirkhart  and  W.  Lynn  Tanner,  "Evaluation 
for  Center  for  Planning  and  Development  of  the  American 
Management  Association."    Report  submitted  to  the  American 


effect  on  local  Educational  Agencies  (LEA's)  will  be 
presented  for  both  years  since  no  evaluation  of  the 
impact  of  training  on  lea's  was  offered  in  the  first 
y oar's  report. 


Management  Association  and  the  United  States  Office  of 
Education,  Syracuse  University,  October,  1971. 
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CHAPTER  ONE 


THE  AMERICAN  Mi^AGEtlENT  ASSOCIATION'S  PROGRAM 
DESIGN  AND  ITS  EVALUATION 


Section  1;    The  American  Management  Association's 
Proposal  of  1970 

The  AMA*s  initial  proposal  rested  on  the  proposi- 
tion that  certain  business  management  practices  and  skills 
could  be  modified  and  effectively  applied  to  education* 
After  some  observation  emd  discussion  with  educational 
administrators,  the  AMA  had  concluded  that: 

•  •  •  most  school  administrators 
need  to  improve  their  knowledge  of 
the  theories  as  well  as  develop 
their  skills  in  applying  the  prac- 
tices  of  management  •  •  •  that 
knowledge  of  the  me£ins,  plus  skills, 
plus  motivation  for  effective  imple- 
mentation of  meems  to  solve  problems, 
are  needed  administrative  capabili- 
ties in  the  nation's  educational 
systems. ^ 

Specifically,  the  proposal  set  forth  two  objectives: 

1)  determining  the  feasibility  of 
developing  and  applying  the  AMA's 
management  development  programs 
to  education  and 

2)  to  introduce  the  educational 
programs  into  representative 
multi-state,  multi-level  groups 
of  educational  administrators #2 


Feasibility  and  Pilot  Program  Proposal:  Adapting 
and  Testing  Business  Management  Development  Programs  for 
Educational  Administrators,"  American  Management  Associa- 
tion, June  22,  1970  (Mimeographed) ,  pp.  2-3. 
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Organizations  involved  in  the  pilot  project  were 
two  state  depeurtments  of  education  and  foiir  local  school 
districts,  two  in  each  of  two  states. 

The  proposal  listed  fourteen  specific  training  goals. 
Each  participating  state  education  agency  would  demonstrate 
to  cm  independent  team  of  reviewers  the  degree  to  which 
these  goals  had  been  reached.    It  was  not  assumed  that  all 
the  goals  would  be  fully  realized  during  the  first  year. 

The  goals  were: 

1)  an  agreed-upon  definition  of  the 
agency's  mission, 

2)  established,  continuing  objectives 
and  planning  procedures  for  long* 
range  achievement  of  the  institu* 
tion's  mission, 

3)  identified  resources  and  constraints, 

4)  differentiated  between  where  the 
institution  is  going  and  where  it 
wants  to  go, 

5)  modified  previously  established 
objectives, 

6)  identified  and  analyzed  alternative 
courses  of  action, 

7)  determined  priorities, 

8)  made  strategic  action  assignments, 

9)  defined  standards  of  performance 
for  key  administrators, 

10)  specified  task  completion  dates, 

11)  designed  supplementary  planning 
efforts, 

12)  assigned  responsibilities  to  sub- 
ordinate units, 

13)  designed  a  metnodology  by  which 
future  performance  may  be  evaluated 
in  relation  to  the  performance 
specified  in  the  plan. 
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14)  produced  and  are  implementing  a 
long-range  strategic  plan*^ 


Section  2;    The  Training  Format 

To  achieve  these  goals  the  Pdik  utilized  a  format  of 
training  developed  over  a  number  of  years  of  work  with 
business  organizations.    The  training  program  package  was 
divided  into  three  distinct  parts  and  adapted  to  the 
special  needs  of  the  educational  agencies. 

Program  components  were  (1)  the  M2magement  Course 
for  Presidents  (MCP) ,  (2)  the  Top  Management  Briefing  (TMB) , 
and  (3)  the  Educational  Planning  Process  (EPP) •  These 
were  presented  in  order  to  each  state  department  of  educa- 
tion/  though  the  local  educational  agencies  were  not 
offered  the  MCP. 

The  three  programs  are  described  below  with  special 
emphasis  on  their  design  and  content* 


1.    The  Management  Course  for  Presidents  (MCP) 


Change 
Target : 

Duration: 

Location: 

Type  of 
Progrcim: 


State  Superintendent  of  Education 
Ah  days 

AMA  Grove  Training  Center  in  Hamilton,  New  York 


Stranger  Training  Situation.    Designed  for  chief 
executives  who  represent  their  organizations. 
Rarely  does  more  than  one  participant  from  the 
same  organization  enroll.    Total  number  of 
participants  is  approximately  24^  including  the 
trainer. 


Ibid.,  pp.  4-5. 

^For  a  more  detailed  discussion  of  the  training 
strategies  employed  by  the  AMA  the  reader  is  directed  to 
the  First  Year's  Evaluation  Report.    Larry  Kirkhart  and 
W*  Lynn  Tanner,  "Evaluation  for  Center  for  Planning  and 
Development  of  the  American  Management  Association.** 
Report  submitted  to  the  American  Management  Association 
and  the  United  States  Office  of  Education,  Syracuse 
University,  October^  1971,    See  especially  Chapter  One, 
"Professionalizing  Management  and  Planning:    A  Strategy 
for  Change." 
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Program 
Goal: 


As  stated  in  the  AMA's  handbook  the  MCP  is 
designed  "...  to  bring  top-management  people 
up-to-date  on  the  status  of  management  as  a 
profession^  and  to  acquaint  them  with  the  kind 
of  formal  education  and  training  that  is  avail- 
able to  the  professional  manager. "5 


Progreun 

Content, 

Learning 

Format, 

and 

Input 

Control: 


Training  Content 


(In  order  presented) 
tH-  Management 
Theory  & 
Practice 

-  Mamagement  as 
a  Profession 

-  Principles  of 
llanagcment 

-  Pattern  of 
iManagement 
Action 

-  najor  skill 
of  Management 

-  Management 
Ethics 

#2-  Communication 

•  Leadership 
Communications 

#3-  Planning 

•  Setting  Corp. 
Objectives 

•  Strategic  & 
Operational 
Planning 

#4-  Organization 
^eory  &  Frac- 
Hce 

Ifanager  Man- 
power Planning 


%  of 
Progremi 
Time 


13.4 


7.1 
14.3 


13,4 


Learning 
Format 


Lectures 
by  AMA 
represen- 
tatives 


Supple- 
mented 
with 
visual 
aids 


Occasion- 
al didac- 
tic in- 
teraction 
with 


Input  Con- 
trolled 
by6 


AMA 


5-1 


ERiC 


'Purpose  of  a  Presidents  Association  Management 
Briefing."    Taken  from  notebook  material  distributed  by 
the  AMA  in  The  Presidents  Association  Notebook/  American 
Memagement  Association,  1970. 

Shis  listing  of  program  content  was  developed  by 
the  First  Year's  research  team,    Kirkhart  and  Tanner,  op. 
cit./  p.  6, 
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lecturer 

or  other 

partici- 

#5- Cllmai:e 

pants 

IvC^vTw  bll   0£    X  op 

M  A  n      ATnon  4* 

It  vjt  DQIUlvf  X 

X  /  •  u 

ue ve  xopxn^  nan*" 

agerial  Standards 

of  Performance 

-  Implementing 

Concept  of 

Professional 

Mgm't. 

#7-  Leadership 

7.1 

^  Assuring  a 

Dynamic  Organi* 

zation 

General  Discussions 

12.5 

Led  by  AMA 

Primarily 

represen- 

AMA 

9  x^  rigm  u  •  rneory 

tative; 

Secondisur- 

orienta- 

ily by 

Fxannxng 

tion: 

client 

client  to 

client 

reactions 

Small  Group  Discussion 

2.7 

LeaderlesSj 

Shared 

task  ori- 

between 

#1-  Planning 

ented 

?MTi  and 

groups. 

client 

Client  to 

client 

reactions 

2.    Top  Management  Briefing  (TMB) 


Change 

Targets:      Program  presented  to  the  most  influential  organiza- 
tional members  of  SEA  and  Local  School  Districts: 
State  Superintendent  and  eleven  of  his  associates/ 
deputies • 

O  The  Director  of  Program  Services  in  each  state  and 

ERIC 
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Duration: 


Location: 

Type  of 
Program: 


Program 
Goals: 


Program 

Content/ 

Learning 

Format/ 

and 

Input 

Control: 


eleven  of  his  subordinates* 

From  each  of  th€.  Local  School  Districts  the  Super- 
intendent and  his  eleven  most  important  advisors • 
Total  personnel  involved:    Approximately  48. 

3^5  days  -  Presentations  made  separately  to  State 
Department  personnel  and  then  to  Local  Educational 
Agency  personnel  in  each  state. 

At  training  site  near  focal  organization 


A  Modified  Diagonal  Program.    The  sessions  were 
composed  of  members  from,  the  same  organization/  but 
different  ranks  and  from  different  departments. 
However  because  of  the  special  nature  of  the  organi- 
zations involved  there  were  cases  where  persons 
from  the  same  department  were  present  at  the 
sessions. 


The  primary  purposes  of  the  TitP  was  to  spread 
knowledge  of  modern  management  practices  throughout 
key  roles  in  SEA  and  Local  School  Districts  and: 


1. 


2. 


3. 


to  motivate  educators  to  consider  the 
application  of  principles/  policies/ 
and  purposes  of  modern  management; 
to  develop  a  practical,  results-oriented 
centrum  of  knowhow  and  experience  upon 
which  administrators  may  draw  to  cope 
effectively  with  current  problems  in 
educational  administration;  and 
to  create  a  means  through  which  adminis- 
trators may  exchange  information  as  to 
the  effectiveness  of  various  approaches 
to  particular  problems  and  minimize 
duplication  of"  experimentation  and 
repetitious  failures. 7 


Training  Content 

%  of 
Program 
Time 

Learning 
Format 

Input  Con- 
trolled 
byS 

#1-  Management  Theory 

13.8 

Lectures 
by  AI>1A 
represen- 
tatives 

;^iA 

i 

s  Practice 
-  Management  as  a 
Profession 

^American  Management  Association,  "Feasibility  and 
Pilot  Program  Proposal,"  o£.  cit. ,  p.  6. 

^Kirkhart  and  Tanner,  0£.  cit.,  p.  9. 
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-  Principles  of 
Management 

-  Pattern  of 
Management 
Action 

-  Major  Skill  of 
Management 

-  Management 
Ethics 

*2-  Planning 

-  Strategic  and 
Operational 
Planning 

#3-  Organization 

Theory  &  Practice 

-  Organizing  the 
Mgmt.  Team 

#4-  Climate 

-  Assuring  Dynamic 
Organization 

-  Growth  of  Man- 
agement Personnel 

#5-  Control 

-  Controlling  the 
Mgmt*  Team 

-  Developing  Mana- 
gerial Standards 
of  PerformanjQ 

#6-  Training 

-  Preview  of 
Strategic  Educa- 
tional Planning 
Process  (to  be 
held  in  Hamilton, 
New  York) 

#7-  Leadership 

-  Styles  of  Leader- 
ship 

-  Application  of 
Mgmt.  Principles 
to  Education 


11.6 


lO.St 


Supple- 
mented 
with 
visual 
aids 


Occasion- 
al didac- 
tic in- 
teraction 
with 
lecturer 
or  other 
partici- 
peoits 


General  Pi  .ussions 

#1-  Group  Leaders 

Reports  on  "Organ- 
izing Mgmt.  Team" 

#2-  Summary  &  Conclu- 
sions: T^!B 

#3-  TMB  Program 
Feedback 


12.0 


Led  by 
AMA 

represen- 
tative? 
orienta- 
tion: 
client  to 
client 
reactions 


Primar- 
ily AMA 
Second- 
arily by 
client 


18 


Small  Group  Discussion 

7.4 

Leader less. 

Shared 

task  ori- 

between 

#1-  Organizing  the 

ented 

At4A  and 

Mgmt.  Team 

groups • 

client 

Orienta-* 

tion: 

client  to 

client 

reactions 

3.    Educational  Planning  Process  (EPP) 


Change 
Targets: 


Duration: 


Location: 

Type  of 
Program: 


Program 
Goals : 


Progrcun  presented  to  Top  Management  groups  who  were 
viewed  as  crucial  to  the  implementation  of  program 
in  each  state: 

State  Superintendent  and  eleven  of  his  associates/ 
deputies. 

The  Director  of  Program  Services  in  each  state  and 
eleven  of  his  subordinates,  and  separately  to  two 
local  school  superintendents  and  eleven  of  their 
subordinates* 

Total  persons  involved:  48. 

Two  5  day  training  sessions  separated  by  at  least 
4  weeks  for  each  of  the  four  groups  in  each  state* 

AMA  Grove  Training  Center  in  Hamilton,  New  York. 


Family  Program.    Composition  of  the  program  parti- 
cipants made  up  of  organizational  leader  and  his 
subordinates. 


Build  an  effective  work  group  which  would  do  the 
following: 


First  Week:  1* 

2. 
3. 

4, 

5, 


develop  a  definition  of  the 
agency's  mission 
analyze  the  agency's  resources 
establish  continuing  objectives 
for  the  agency 

develop  specific  objectives  and 
set  priorities 

determine  informational  needs  to 
evaluate  alternative  courses  of 
action 
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Program 

Content # 

Learning 

Format  r 

and 

Input 

Control: 


6. 


Second  Week:  1, 


2. 
3. 

4. 
5. 


assign  specific  data  gathering 
tasks  and  sot  due  dates. 

define  planning  gaps  -  the 
difference  between  where  the 
agency  is  2md  where  it  wants 
to  go 

modify  preliminary  objectives 
analyze  alternative  courses  of 
action  open  to  the  agency 
finalize  priority  decisions 
develop  specific  action  assign- 
ments and  supporting  standards 
of  performance  and  estimated 
times  of  completion*^ 


Training  Content 


#1-  Mission 

«  Organizations 
function 

-  Justification 
for  Continued 
Societal 
Support 

-  Focus  for  Re- 
sources to  bo 
Applied 

#2-  Internal  Analysis 

-  Nature  of 
Institution 

-  Capability  & 
Limitations 

-  Factors  under 
Institutional 
Control 

-  Organization 

-  Beliefs 

-  Characteris- 
tics 

-  Functions 

-  Resources 

-  Strengths 

-  Weaknesses 


%  of 
Program 
Time 


Variable 
and  sit- 
uationalf 
depend- 
ing upon 
problems 


All 

areas  to 
be 

covered 
as  cOTi- 
pletely 
as  pos- 
sible by 
end  of 
week. 


Learning 
Format 


Parameters 
of  learn- 
ing pro- 
cess, 
steps  and 
procedures 
defined 
by  AMA* 
Client 
learning 
to  occur 
through 
inter- 
personal 
interac- 
tion- 
client  to 
client 
and  cli- 
ent tc 
AMA  re- 
source 
person. 
All 

learning 
outcomes 


Input  Con- 
trolled 
bylO 


AMA  and 
client 
(Discus- 
sion of 
boundar- 
ies and 
depth 
largely 
controlle<  [ 
by  AMA 
represen- 
tative; 
specific 
problems # 
documents, 
organiza- 
tional 
contexts , 
and  or- 
ganiza- 
tional 
processes 
defined 
by  client) 


ERLC 


Aim,  "Feasibility  and  Pilot  Program  Proposal,"  0£. 
cit. ,  pp.  7-11. 

^^Kirkhart  and  Tanner,  o£.  cit. ,  pp.  14-15. 
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#3-  External  Analysis 

-  Factors  Outside 
the  Control  of 
the  Organiza- 
tion (i.e.  ex- 
ternal con- 
straints) 

•  -  Trends 

-  Rate  of  Change 

-  Kind  of  Change 

-  Assumptions 
About  Future 

#4-  Objectives 

-  Desired  Results 
or  Ends 

-  Quantitative 
Terms 

-  Specify  Means 
for  Evaluation 

-  Short,  Long 
Range  or  Con- 
tinued Objec- 
tives 

-  Differentia- 
tion of  Ends 
from  Means 

^5-  Intersession 
Assignments 

-  Data  rJecessary 
to  Development 
of  Strategies 
to  Accomplish 
Each  Objective 

-  Historical 
Perform.  Data 

-  Cost  Data  for 
Coat/Benefit 
Analysis  of 
Alternatives 

-  Data  Necessary 
for  Organization- 
al Evaluation 

f 

are  1 
respc 
sibi] 
of  « 
clier 
(not 
AMA). 

the 

Dn- 

Lity 

le 

It 

Intersossion  Break:    Return  to  Organization  for 

Minimum  of  four  weeks 

#6-  Priorities 

-  Priority  Ranking 
of  Objectives 
Based  on  Team 
Evaluation  of 

V 

f 

f 
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5  Resource  Alio- 
E  cation 

C  -  Sense  of 

0  Urgency 

N  •  Cost  of  Imple- 

D  mentation 

-  Probability 
of  Success 

T  -  Long-term  vs* 

R  Short-term 
A  Benefit 

1  -  Public  Demands 
N  #7-  StratecLies 

I        -  Means  to  x^chieve 
M  Results  Specified 

6  in  Objectives 

-  Development  of 
Iter  natives 

P  -  Cost/Benefit 

R  Analysis  Before 

O  Strategy 
G  Decision 
R  #8-  Programs 
A        -  Specific  Results 
M  «  Delegation  to 

a  Person 

-  /acceptable  Tar- 
get Date  (mutu- 
ally agreed) 

-  Sum  of  all  Pro- 
grcuns  Equal  to 
Results  Antici- 
pated in  Specific 
Objectives 

f^9-  Planning  Schedule 
-  Schedule  of  Events 
&  Targot  Dates 

-  Insure  Planning 
is  "Way  of  Life" 


Section  3:    Tho  AMA  Strategy  for  Change 

In  analyzing  the  AIlA's  training  process  and  its 
potential  impact  on  organi'^ational  systems  /  two  factors 
were  thought  critical*    First/  the  type  of  training 
program  was  viewed  as  affecting  the  orograia's  effect  on 
organisational  beh^ivior  and  goal  attainment*    Second/  how 
the  program  was  implemented  reflected  basic  beliefs  about 
attitude  change  and  in  turn  affected  the  possibility  of 
changing  trainee  attitudes 


22 


The  Yl  research  team  determined  that  the  prcgreuna 
was  implcunented  primarily  by  meana  of  attitude  change 
through  the  trainees*  compliance  and  identification  v;ith 
concepts/  experiences^  and  values  presented  by  the  AHArl2 
This  method  may  be  contrasted  with  seeking  attitude  change 
through  internalization  of  change. 

The  methods  of  input  control  used  in  the  training 
indicated  that  if  any  attitudes  were  to  be  changed,  they 
would  alter  through  compliance  and  identification.  As 
indicated  in  the  display  of  the  training  format  above, 
approximately  80%  of  program  time  was  controlled  by  the 
AMA  lecturer.    In  this  context,  for  change  to  occur,  the 
training  participants  must  comply  and  identify  with  the 
concepts,  experiences,  and  values  presented  by  the 
trainer.    As  the  first  year's  evaluation  pointed  out,  this 
process  can  be  termed  an  informational  method  of  change,  a 
method  heavily  dependent  on  the  quality  of  the  information 
imparted  to  the  client  and  on  the  client's  need  for  that 
specific  information. 

The  Educational  Planning  Process  (EPP)  program 
differed  from  the  IICP  and  TIIB  by  dispersing  control  over 
program  input.    However,  the  primary  means  of  obtaining 
attitudinal  change  was  again  defined  as  compliance  and 
identification.    Although  the  EPP  input  was  largely 
determined  by  the  client  group,  who  v;ere  invited  to 
discuss  organizational  issues,  goals,  interaction  of 
administrators  with  each  other,  etc.,  the  trainer  acted 
to  establish  and  clarify  the  boundaries  of  legitimate 
discussion  and  insured  that  each  step  of  the  planning 
process,  as  defined  by  the  i\MA,  was  accomplished  as  fully 
as  possible.    An  attempt  was  made  to  maximize  the  level 


^^Ibid.,  p.  127. 

^^In  analyzing  the  AMA's  approach  toward  attitude 
change,  the  First  Year's  research  team  developed  a  typology 
of  attitude  change  composed  of  compliance,  identification # 
and  internalization.    Compliance  was  defined  as  learning 
to  say  or  do  the  expected  thing  regardless  of  the  indi- 
vidual's private  beliefs.    Change  through  identification 
was  defined  as  acceptance  both  privately  and  publicly 
v;hich  is  evoked  when  the  individual  is  acting  within  the 
relationship  upon  which  the  identification  is  based. 
Change  through  internalization,  on  the  other  hand,  was 
defined  as  the  acceptance  of  influence  from  outside  forces 
because  the  nature  of  the  influence  is  congruent  with  the 
person's  value  system.    These  concepts  were  drawn  from: 
Herbert  C.  Kelman,  "Processes  of  Opinion  Change, •*  Public 
Opinion  Quarterly,  XXV  (Spring,  1961),  pp.  57-78.    Ibid . , 

ERLC 
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of  rational  dialogue  2md  exchemge  of  opinion.  Because 
the  possibility  of  internalization  of  attitude  change  is 
optimized  when  both  logical  discussion  and  affect  ore 
considered,  the  first  year  research  team  did  not  believe 
that  internalized  attitude  change  would  be  produced  in 
the  EPP. 15   However,  because  this  family  or  team  program 
Involved  an  organizational  superior  and  his  subordinates, 
any  changes  due  to  the  training  program  were  viewed  as 
products  of  the  EPP's  impact  on  role  relationships  and 
group  standards. 

Based  on  an  analysis  of  five  different  types  of 
training  programs  the  Yl  research  team  classified  the  AI-IP. 
programs  as  follows: 16 


Change  Emphasis  of  ABfA  Program 

Type  of 

Training 

Program 

Individual 

Awareness 

Knowledge 

Inter-*Personal 
Expectations  of 
Role  Relations 
&  Group 
Standards 

Inter*6roup 
Standards  of 
Appropriate 
Relation- 
ships 

Stremgor 

Program: 

Primary 

Diagonal 

Progtamt 
tmS^  

Primary 

Tertiary 

Secondary 

Family 

Program: 

EPP 

Secondary 

Primary 

Tertiary 

17 


^^Kirkhart  and  Tanner,  o£.  cit. /  p.  128. 
^^Ibid. 

^^Other  types  of  training  programs  analyzed  were  the 
Cousin  Program  which,  like  those  mentioned  in  the  text,  was 
adopted  from  Warren  Bennis,  Changing  Organizations  (New 
York,  1966),  pp.  120-121  and  the  Inier-uepartment  Program 
adopted  from  Jack  Fordyce  2md  Raymond  Weil,  Managing  With 
People  (Reading,  Mass.,  1971),  pp.  124-130.    "big.,  ppTTl-33. 

^^This  chart  was  adopted  from  the  First  Year's 
report.    Ibid.,  p,  35. 
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In  the  stranger  program  (MCP) ^  potential  organiza- 
tional change  depends  on  the  capacity  of  a  single  person 
to  return  to  the  organization  and  change  it.    In  the 
diagonal  program  (TMB) ^  most  of  the  potential  for  change 
is  based  on  the  capacity  of  individuals  who  return  to 
highly  influential  roles  in  the  organization  and  effect 
changes  based  on  their  new  knowledge  and  awareness.  The 
faunily  program  (EPF)  seemed  most  promising  since  one  goal 
of  the  program  was  to  cause  changes  in  expectations  aJbout 
role  relationships  and  group  standards  of  behavior  among 
the  management  teams  involved  in  the  training. 

Based  on  the  assumption  that  the  likelihood  of 
organizational  impact  is  related  to  training  design  and 
the  number  and  types  of  people  who  trained,  the  Yx  research 
team  concluded  that  the  greatest  impact  and  the  linkages 
between  the  training  and  organizational  change  were  in  (1) 
individual  awareness/knov/ledge  and  (2)  interpersonal  ,g 
expectations  of  role  relationships  and  group  standards. 
These  variables  are  the  channels  for  transmitting  change 
which  the  AEIA  program  depended  upon  to  introduce  and 
support  more  effective  planning. 


Section  4;    The  First  Yearns  Evaluation  (Yl) 


For  the  purposes  of  the  Yl's  evaluation  the  research 
team  placed  primary  emphasis  on  measuring  the  o  itcomes  of 
the  Educational  Planning  Process,  a  decision  taJcen  for 
several  reasons.    First,  the  EPP  training  program  objec- 
tives were  most  directly  related  to  the  outcome  goals  stated 
by  the  ?2ih  for  the  entire  pilot  project. (See  page  12 
of  this  report  for  a  list  of  these  outcome  goals.) 

Second,  the  EPP  was  designated  as  the  "...  most 
potent  force  of  organizational  change  .  .  ."in  the  program 
packages  offered  by  the  AMA.20    This  program  was  the 
beneficiary  of  whatever  change  occurred  in  the  MCP  and 
TM  .    In  addition,  the  EPP  program  format  gave  participants 
time  to  return  ro  their  organizations  to  continue  the 
processes  begun  d^^ring  the  first  week  of  the  EPP.  This 


Ibid. ,  p.  36. 

19 

See  page  12  of  this  report  for  a  list  of  these 
outcome  goals. 

20 

Kirkhart  and  Tanner,        'cit. ,  p.  129. 
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procedure  provided  the  greatest  potential  for  effecting 
change  in  the  organizations. 21 

Finally,  as  a  family  program  the  EPP  seemed  most 
likely  to  encourage  change  through  the  conceptual  linkages 
of  (1)  role  relations  and  group  standards  and  (2)  individ- 
ual awareness Anowledge.    For  change  in  the  organizations 
to  occur,  it  would  have  to  be  transmitted  through  changes 
in  these  attitudes. 22 

Thus,  based  on  the  analysis  of  the  training  design 
cind  methods  of  implementation/ the  Yl  evaluation  centered 
on  the  problem  of  measuring  (1)  changes  in  individual 
awareness/knowledge  and  <2)  changes  in  interpersonal  expec- 
tations of  role  relationships  and  group  standards.  The 
measures  of  these  areas  were  related  to  the  goals  of  train- 
ing in  order  to  assess  the  impact  of  the  program. 

Since  the  ATIA  program  had  been  completed  only  a  short 
time  before  the  Yl  evaluation,  the  research  team  felt  it 
was  inappropriate  to  explore  the  question  to  what  extent 
the  plans  prepared  during  the  EPP  wore  being  acted  upon 
in  the  experimental  organizations. 

We  emphasized  the  atibitudinal  change  which  would  be 
a    necessary  but  not  sufficient  basis  for  changing  the 
planning  process  in  the  organizations.    Unless  attitudes 
shifted  to  emphasize  organizational  planning,  and  in  a  way 
aligned  with  the  AMA's  approach,  it  would  be 
difficult  to  foresee  that  behavior  and  ultimately  organi- 
zational output  would  change  in  order  to  implement  the 
plans  set  forth  during  the  EPP.    To  measure  the  impact  of 
the  training  on  the  attitudes  of  the  participants,  the 
research  tecim  employed  Likert's  concept  of  causal  and 
intervening  variables. 23 


'^•"•Daniel  Katz  and  Robert  Kahn,  The  Social  Psychology 
of  Organizations  (New  York,  1966),  pp.  390-391.  " 

22 

*  The  First  Year's  Evaluation  stressed  the  central 
importance  of  linkages  in  producing  change  and  the  differ- 
ence between  linkages  and  program  content.    For  change  to 
occur,  its  preconditions  must  be  transmitted.  Training 
program  content  is  important,  but  is  obviously  futile  if  it 
is  not  transmitted  to  the  change  targets  in  a  manner  which 
insures,  or  at  least  increases  the  probability,  that  they 
will  adopt  it.    Kirkhart  -^nd  Tanner,  0£.  ext.,  p.  36. 

Rensis  Likert,  The  Human  Organization  (New  York, 
1967),  pp.  28-29. 
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The  causal  variables  were  organized  into  a  broad 
category  called  Organizational  Planning >    These  independent 
variables  can  be  altered  or  changed  by  the  organization 
and  its  management  and  require  individual  knowledge/aware- 
ness.   In  the  Yl  report  they  were  divided  into  sub-sets 
as  follows:     (1)  Definition  of  the  Mission  of  the  Organi- 
zation, (2)  Mobilization  of  Organizational  Plemning,  (3) 
Operational  Impact  of  Training  on  Organizational  Plemning, 
and  (4)  Credibility  of  the  Planning  Process •    These  sub- 
sets contained  22  research  variables, all  related  to  the 
stated  goals  of  the  hMH  for  the  Organizational  Planning 
Process. 

A  second  broad  category  was  established  to  reflect 
Likert's  concept  of  intervening  cariables.  Registering 
the  internal  state  and  health  of  the  organization,  these 
variables  touched  upon  what  the  AMA  called  professional 
management,  i«e«,  leadership,  control,  motivation,  etc. 
The  category  of  variables  was  called  Role  Relationships 
and  Group  Standards  by  the  Yl  evaluation.    It  was  divided 
into  three  sub- sets:     (1)  Leadership,  (2)  Decision-Making, 
and  (3)  the  Management  Team.    These  sub-sets  contained 
17  research  varicibles.^^    These  variables  were  related  to 
the  process  of  managing  the  organization  and  were 
presumed  to  be  affected  by  the  TUB. 

Given  tfte  design  of  the  MIA  program  and  its  emphasis, 
if  organizational  change  occurred  we  concluded  that 
measurable  changes  must  have  occurred  within  these  broad 
categories. 

In  order  to  provide  a  valid  basis  of  assessment  of 
the  effects  of  training,  the  research  team  added  a  control 
group  to  the  organizations  to  be  studied,  a  comparable 
state  department  of  education.    A  Non-Equivalent  Control 
Group  research  design,  to  be  discussed  in  detail  in  Part 
II  of  this  report,  was  thus  established. 


Section  5s    Summary  of  the  First  Year's  Evaluation  Findings 

The  findings  reported  after  the  Yl*s  evaluation 
concerned  only  the  two  state  departments  of  education-- 
referred  to  here  as  State  El  and  State  E2. 

The  overall  conclusion  of  the  research  team  was 
that       .  .  the  ej^fect  of  training  during  t"he  j^irst  year 


24 

Kirkhart  and  Tanner,  0£.  cit . ,  pp.  39-40. 
^r.^^  ^^Ibid. 
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of  evaluation  was  very  limited  in  experimental  State  El 
and  limited  in'llxperimental  State  E2»^b 


Area  1:    Organizational  Planning 

No  significant  changes  were  registered  in  either 
organization  in  terms  of  the  V2u:iable8  set  to  measure 
changes  in  defining  the  mission  of  the  organization  or  in 
mobilizing  organizational  planning.    In  fact,  seven  cases 
of  negative  effects  were  measured.    The  data  suggest  that, 
at  the  time  of  measurement,  no  positive  effects  regarding 
the  specific  training  goals  had  emerged. 

There  were  positive  effects  experienced  in  State 
E2  under  the  sub-set  of  variables  dealing  with  the  operation- 
al impact  of  organizational  planning.    The  participants' 
perceptions  changed  in  a  positive  direction  concerning  the 
operability  of  the  organization's  overall  plan,  goals,  and 
policy  statements. 

The  measurement  of  veuriables  related  to  the  credibil- 
ity of  planning  also  showed  no  effect  for  either  state 
except  that  State  E2  did  show  a  positive  effect  toward  the 
role  of  planning  in  the  SED. 


Area  2:    Role  Relationships  and  Group  Standards 

An  analysis  of  the  variables  related  to  changes  in 
Role  Relationships  and  Group  Standards  revealed  only  seven 
areas  of  positive  effects  out  of  a  total  of  18  variables 
measured. 

The  apparent— though  limited — positive  effect  upon 
State  E2  as  compared  with  State  El  was — in  the  considered 
judgment  of  the  Yl  research  team— the  result  of  the  unin- 
tentional development  of  two  different  training  designs.'*' 

This  develoianent  occurred  in  implementation  of  the 
Educational  Planning  Proce&s.    The  EPP  was  designed  to  be 
presented  first  to  each  State  Superintendent  and  to  his 
eleven  deputies/associates  over  a  period  of  two  weeks.  The 
program  was  divided  into  two  segments  by  a  four  week 
intersession  between  the  first  and  second  weeks  of  the 
EPP.    A  second  EPP  was  then  scheduled  to  be  administered 


28 


to  erich  Deputy  Super intondont  in  charge  of  program 
Services  and  to  eloven  of  his  subordinates.    This  was  the 
manner  of  implementing  the  program  in  State  El. 

In  State  E2  however^  it  was  decided  to  expand  the 
first  EPP  by  one  week  because  more  time  was  required  to 
complete  the  training  process.    There  was  also  a  50% 
overlap  of  personnel  between  the  Superintendent's  group 
and  that  of  the  Deputy  for  program  service's  group.  Thus, 
the  first  group  in  State  E2  took  three  weeks  of  the  EPP, 
returning  to  Hamilton  twice. 

As  State  El  did  not  undergo  an  identical  training 
period,  the  First  Year  research  team  determined  that  a 
different  training  design  had  been  used,  and  explained  the 
slight  difference  in  terms  of  positive  effects  observed  in 
the  experimental  states. 


Section  6;    The  Present  Evaluation  Project:    Focus  and 
Intent  of  the  Second  Yearns  Evaluation 

Assessing  Management  Training 

Judging  the  effectiveness  of  managerial  training 
is  an  important  part  of  the  overall  evaluation  of  efforts 
to  improve  organizational  performance.    The  part/  like 
the  whole,  certainly  needs  improvement.    We  lack  reliable 
evidence  of  the  extent  to  which  training  programs  induce 
positive  changes  in  participants'  attitudes  and  actions. 
In  the  present  evaluation  we  have  attempted  to  evaluate 
the  impact  of  training  in  both  respects. 

Area  1; 

Concerning  attitudes,  an  evaluation  which  focused 
on  attitudes  observed  during  the  training  process  may 
sometimes  undorestimato  and  sometimes  exaggerate  the  impact 
of  the  program.    Some  effects  appear  later.    Others  persist 
relatively  Intact  for  a  lifetime.    Residual  effects  often 
become  dissipated  by  maturation,  experience,  and  ' 
plain  forgetfulness.    It  is  misleading  to  assume,  there- 
fore, that  attitudes  measured  by  an  immediate  on-site 
postevaluation  will  remain  at  similar  levels  for  long; 
To  base  a  judgment  of  "success"  or  "failure"  on  such 
short-term  data  is  premature. 29 


John  A.  Rehfuss,  "Executive  Development:  Execu- 
tive Seminar  Style,"  Public  Administration  Review,  XXX,  5 
^  (September/October,  lb16) ,  p.  556. 

ER^  ^^See  Warren  G.  Findley,  "Measurement  and  Research  in 
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This  comxnon  practice  was  carefully  avoided  by  the 
Yl  evaluation  team*    Rather  than  gather  data  at  the  train- 
ing site,  the  team  assembled  data  before  the  trainees 
left  the  organization  as  well  as  after  they  returned  to 
the  organization  and  became  immersed  in  their  normal 
organizational  roles.    One  of  the  objectives  of  Y2  evalua- 
tion was  to  extend  the  time  frames  of  the  Yl  effort  to 
understand  more  fully  the  longer  term  effects  of  the  AtlA's 
training  on  particlpeuit  attitudes. 


Area  2: 

On  the  action  side  of  the  evaluation  ledger,  a 
longitudinal  perspective  is  very  important*    Specif ically, 
we  need  to  know  as  much  as  possible  about  the  effects  of 
time  and  the  environment  on  the  plans  produced  during  AMA 
training  in  order  to  assess  more  effectively  the  impact 
of  the  program* 

The  propc  3d  evaluation  is  based 
on  the  pri  mise  that  a  thorough 
evaluation  of  the  pilot  training 
project  and  its  implications  for 
the  improvement  of  the  management 
of  public  institutions  requires  an 
understanding  of  its  impact  over 
time  on  the  participants  and  their 
educational  systems*    Vfhat  happens 
to  the  peurticipemts  and  the  newly 
developed  planning  systems  beyond 
the  initial  training  period?  Are 
they  able  to  make  a  significant 
improvement  in  the  management  of 
the  educational  system?    What  are 
the  unforeseen  problems  and/or 
consequences  which  only  become  ap- 
parent with  time?  •  •  •  In  short, 
vrtiat  is  the  impact  of  the  AMA's 
pilot  project  in  thi  real  world? 


the  Service  of  Education,"  address  to  Division  D,  Measure- 
ment and  Research  Methodology,  AERA  (March,  1970),  p.  6ff. 

^^Kent  J*  Chabotar  and  Stephen  H*  Montgomery, 
••Second  Year  Evaluation  of  USOE/AMA  Pilot  Program:  Adapt- 
ing and  Testing  Business  Management  Development  Programs 
for  Educational  i^dmlnistrators,**  April  10,  1971  (mlmeo)  • 
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In  this  second  area  we  paid  particular  attention 
to  plan  effectiveness^  the  extent  to  which  the  nutput^^ 
of  the  planning  system32  and  the  entire  educational 
agency  achieves  the  continuing  objectives  and  specific 
objectives  of  the  plans  developed  during  emd  eifter  AMA 
training.    In  this  context  we  are  specifically  interested 
in  what  we  have  defined  as  two  levels  of  the  planning 
system's  output. 

Intermediate  Output: 

The  intermediate  output  of  the  planning  system  is 
defined  here  as  the  plans  produced  in  the  system  after 
training  by  the  AMA.    Selected  plans  will  be  evaluated  by 
criteria  that  express  a  '*good*'  plan* 

Final  Output: 

We  have  defined  final  output  as  actual  measured 
progress  toward  the  specific  and  continuing  objectives 
stated  in  the  planning  documents « 

Each  of  these  seems  equally  importzmt  to  us. 
Well  developed  plans  are  an  essential  precondition  of 
the  achiev^ent  of  strategies  and  objectives.    Plans  axe 
merely  meaningless  paper  if  they  do  not  lead  to  the 
measured  attainment  of  goals  and  objectives. 

Finally,  since  this  evaluation  primarily  proposes 
to  examine  the  impact  of  the  orgamizational  environment 
on  the  planning  approach  developed  by  the  AMA,  we  will 
describe  changes  proposed  by  the  participemts  for  their 
planning  system. 

Within  both  areas,  attitudes  and  organizational 
action,  we  have  studied  the  impacts  of  the  training  on 
selected  variables,  not  only  on  the  two  experimental 
state  education  dep2u:tments  (SEO's)  but  also  on  two. 
participating  local  education  agencies  (LEA^s)  in  each 
state. 
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Defined  as  the  plans  and  planning  services  made 
available  by  the  pl2mning  system  to  other  elements 
internal  and  external  to  the  agency. 

32 

Defined  as  the  set  of  interrelated  elements 
within  the  educational  agency  which  combine  to  produce 
planning  output. 
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Our  examination  covers  a  wide  range  of  instruments 
from  interviews  to  observations,  and  from  open-ended  to 
scaled  response  questionnaires.    Its  conclusions  will  be 
strengthened  by  comparing  the  results  in  the  experimental 
SED's  and  LEA's  with  those  in  agencies  which  did  not 
receive  the  same  training  at  the  same  time  or  received 
none  at  all*    Those  "comparison"  groups  have  been  drawn 
from  two  additional  SED's  and  from  another  LEA  in  each 
experimental  state*    'lore  about  this  in  our  next  section 
on  methodology. 
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PART  TWO 
RESEARCH  DESIGN  AND  METHODOLOGY 
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INTRODUCTION 


Research  design  is  a  critical  factor  in  evaluating 
the  effects  of  any  program.    Next  to  determining  what  we 
want  to  study r  how  we  want  to  study  it  is  the  most 
important  question  to  be  answered  by  the  researcher. 
Design  establishes  the  theoretical  framework  for  the 
conduct  of  organizational  inquiry  by  suggesting  what 
observations  to  maker  how  and  when  to  make  them^  and  what 
use  will  and  can  be  made  of  the  data  thus  collected. 
More  importantly r  it  affects  the  degree  to  which  the  data 
can  be  unambiguously  interpreted^  i.e.,  it  attempts  to  • 
control  for  multiple  explanations  of  research  findings. 

In  Part  II  of  this  report  we  describe  the  two- 
year  research  design  used  to  evaluate  the  AMA's  training 
program •    V7g  intend  to  enumerate  the  various  threats  to 
the  internal  and  external  validity  of  the  design ,  and 
the  extent  to  which  our  design  has  controlled  them.  In 
addition r  we  set  forth  a  conceptual  linkage  between  the 
Yl  and  Y2  evaluations.    We  describe  the  variables  studied 
by  the  evaluations r  the  variables  continued  from  the 
first  year's  study,  and  the  new  variables  which  have  been 
added  by  the  present  research  team.    Finally r  we  account 
for  the  instruments  used  to  measure  the  variables:  their 
purpose  and  method  of  construction  as  well  as  the  manner 
in  which  they  were  administered.    The  statistical  tests 
applied  to  the  data  are  also  reported. 
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CHAPTER  TWO 


RESFARCH  DESIGN 


Section  1:    Problem  of  the  Research  Model 

A  wide  range  of  means  to  evaluate  organizations 
is  available;  theoretically^  there  arc  as  many  research 
designs  as  there  are  research  variables.    Since  no 
constant  criteria  exist  for  what  constitutes  good  evalua- 
tion/ a  research  design  is,  in  large  measure #  influenced 
by  the  particular  setting  in  which  it  is  used.    It  cannot 
stand  alone  but  must  address  the  problem  under  study.  It 
provides  certain  kinds  of  information  under  certain  kinds 
of  conditions. 

Perhaps  the  greatest  challenge  social  scientists 
face  lies  in  the  commonplace  nresumrtion  that  evaluations 
must  always  fit  the  Procrustean  bed^  of  the  scientific 
method/  specifically/  the  laboratory  approach.  The 
laboratory  approach  is  a  research  study  in  vrhich  the 
variance  of  all  or  nearly  all  of  the  po^^sible  independent 
variables  not  pertinent  to  the  immediate  problem  of  the 
investigation  is  kept  to  a  minimum.    This  is  accomplished 
by  isolating  the  research  in  a  physical  situation  apart 
from  the  routine  of  ordinary  living  and  by  manipulating 
one  or  more  independent  variablcas  under  vigorously 


Cited  in  a  letter  from  Robert  Palazzi/  President 
of  the  California  Association  of  Teacher's  of  English/  to 
State  Representative  John  Vasconcellos/  Chairman  of  the 
Joint  Committee  on  Educational  Goals  and  Evaluation  of  the 
California  Legislature  (16  June  1971):  • 

Allov;  us  to  remind  you  of  the  legendary 
Greek  character  Procrustes/  a  robber  who 
placed  each  of  his  victims  on  an  iron  bed* 
if  the  victim  was  shorter  than  the  bed/ 
Procrustes  stretched  him,  even  if  it  meant 
breaking  his  joints/  until  he  fitted  it; 
if  the  victim  was  longer  than  the  bed, 
Procrustes  lopped  off  the  overhanging  parts 
of  the  victim's  body.    Thus  attempts  to  re- 
duce me  to  one  standard/  to  one  way  of 
thinking/  or  to  one  way  of  acting  has  since 
been  referred  uo  as  'placing  them  in  a 
Procrustean  bed. • 
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specified,  operationalized,  and  controlled  conditions.^ 


Laboratory  Experiments 


Because  it  does  provide  relatively  complete 
control  over  extraneous  influences  that  nay  affect  analy- 
sis (through  emphasizing  a  high  degree  of  specificity  in 
the  operational  definitions  of  variables,  and  minimiza- 
tion of  error  variance  through  precise  measuring  instru- 
ments and  random  selection  of  treatment  groups),  the 
xaboratory  approach  has  the  virtue  of  being  replicable 
m  other  contexts.    Hence  it  appeals  considerably  to  the 
social  science  researcher  interested  in  making  what  he 
considers  to  be  an  "objective,"  "rigorous,"  and  "univer- 
sally applicable"  contribution  to  his  discipline. 


Field  Experiments 


But  people  in  organizations  are  not  inert  elements 
m  test  tubes.    Their  actions  and  attitudes  are  not 
amenable  to  exact  definition,  external  manipulation, 
transfer  to  other  contexts  or,  more  importantly,  isolation 
from  extraneous  variables.    So  the  order  of  the  laboratory 
IS  exchanged  for  the  disorder  of  the  field  study.  Field 
studies  are  scientific  inquiries  aimed  at  discovorina  the 
relations  and  interactions  among  sociological,  psychologi- 
cal, and  educational  variables  in  actual  social  struc- 
tures.-*   However,       much  as  some  might  hate  to  admit  it, 
the  field  setting  often  represents  reality  for  the  social 
scientist, and  reality  is  the  ultimate  test  of  a  program 
or  policy.    For  of  what  use  is  an  evaluation  program  if 
it  can  be  replicated  only  in  a  laboratory? 

Thus  organizational  evaluations,  including  the 
present  one,  are  not  designed  to  produce  universally  valid 
information  or  new  knowledge  but  data  that  is  valid  and 
useful  within  the  decision-making  context.  Evaluations 
must  generate  findings  and  analysis  relevant  to  particular 
organizations  at  a  particular  time,  no  more  and  certainly 
no  less.    This  exploratory  type  of  field  study  aims  at 
describing  what  is  rather  than  to  predict  relations  to  be 


Fred  N.  Kerlinger,  Foundations  of  Behavioral 
?A>^f'^°^  (New  York:    Holt,  I^inehart  and  Winston,  Inc., 
1964),  p.  379. 
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found, ^    Field  studies  have  throe  basic  purposes:  to 
decide  on  signific^int  varir?.bles  in  the  research  situation, 
to  discover  relations  among  varieibles,  and  to  suggest 
guidelines  for  future  research,^ 

All  of  this  is  not  to  suggest  that  the  laboratory 
approach  is  necessarily  an  inferior  research  model  in 
social  systems,  nor  do  we  idealize  the  field  study 
approach.    Neither  attitude  is  easy  to  adopt  for  those 
familiar  with  either  approach,  although  it  might  be  for 
people  who  lack  a  basic  understanding  of  research  methods. 
Each  approach  has  advantages  and  liabilities  depending  on 
the  circumstances  being  studied,  and  each  must  be  evaluated 
accordingly. 


Section  2:  Provisional  Research  Design!  Promise 

and  Performance^ 

Vlhen  the  research  team  was  first  introduced  to  the 
AMA  training  program  in  1970,  it  soon  became  clear  that 
the  only  explicit  assumption  which  had  been  made  about 
our  evaluation  was  that  "before  and  after"  data  would  be 
gathered  from  the  two  state  ^=iducation  departments  (SED's) 
which  had  committed  themselves  to  the  program.  Conclusions 
as  to  its  effects  on  participants  and  the  organizations 
of  vrhich  they  were  a  part  were  to  be  based  on  data 
collected  by  this  method. 

The  general  concept  for  evaluation  of 
almost  any  learning  experience  is 
fairly  straightforward;  its  applica- 
tion more  complex.    Basically  our 
approach  is  to  conduct  "before  and 
after"  evaluation  studies.' 


^D.  Katz,  "Field  Studies,"  in  L.  Festinger  and 
D.  Katz,  Research  Methods  in  the  Behavioral  Scir^ces  (New 
York:    Holt,  Rinehart  and  T7inston,  Inc.,  1953),  pp.  75-83. 

^Ibid. ,  p.  75. 

^ilaterial  under  this  heading  v;a^  adopted  from  the 
Kirkhart  nnd  Tanner  report,  pp.  19-23. 

^Treadway  C.  Parker,  "Suggestions  Concerning 
Evaluation  of  the  AMA/USOE  Training  Project"  (Hamilton, 
New  York:    AtV*,  6  Aug.  1970). 
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In  the  litoraturo,  this  approach  to  evaluation  is 
called  a  "One  Group  Pretest  Posttest  Dooign"  duolicated  in 
two  crganizations.^    At  first  glance  this  seems  to  be  a 
satisfactory  way  to  assess  the  training  nrograip.  The 
results  of  a  "before"  test  could  be  compared  with  an 
"after"  test  and^  if  the  differences  were  statistically 
significant,  could  indicate  a  positive  or  negative  effect 
of  the  training. 

However,  from  the  standpoint  of  research  methodology, 
this  provisional  design  has  serious  drawbacks.  Data 
gathered  by  this  method  is  susceptible  to  a  variety  of 
multiple  explanations. 5    In  other  words,  we  do  not  know 
whether  or  not  observed  pre/post  chcinges  are  the  result 
of  the  training  prograun  or  some  other  factor. 

Because  of  the  widespread  use  of  this  design  in 
educational  research  and  training  evaluations,  the  inherent 
limitations  of  the  AliA- suggested  design,  and  the  effect 
of  its  modification  on  strengthening  the  subsequent 
evaluation,  it  is  critical  that  we  clccirly  explain  the 
consequences  of  utilizing  this  "One  Group  Pretest  Posttest 
Design. 

Before  beginning  the  explanation,  some  definition 

of  the  symbols  to  be  used  is  necessary.    In  this  analysis 

we  shall  rely  on  an  adaptation  of  the  work  of  Camobell  and 
Stanley. 10 

X  =  experimental  or  treatment  variable 

0  «  observation,  data  gathering  at  a 
specific  point  in  time 

R  at  the  end  of  a  row     random  assignment 
of  subjects  to  experimental  groups 
and  the  random  assignment  of  experi- 
mental treatments  to  experimental 
groups 


Donald  Campbell  and  Julian  Stanley^  Experimental 
and  Quasi*-Experimental  Designs  for  Research  (Chicago: 
Rand  McNally  S  Co.,  1966),  p*  7. 

^Ibid. ,  pp.  9-12. 

^^Ibid. ,  p.  6.    See  also:    Kerlinger,  o£.  cit. ,  pp. 
292-293;  N.  Gage  (ed.).  Handbook  of  Research  on  Teaching 
(Skokie,  Illinois:    Rand  McNally,  1963) . 
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Series  of  dashes  (  )  between  rows  =  a 

nonremdom  assignment  of  subjects  to 
experimental  groups  and/or  the  non- 
random  assignment  of  experimental 
treatments  to  experimental  groups. 

Any  X*s  or  O's  in  a  given  row  indicate  the  application  of 
these  variables  to  the  same  group*    Moving  from  left  to 
right  represents  forward  movement  from  one  point  in  time 
to  another*    Vertical  assignments  of  these  symbols  indicate 
that  these  events ^  either  X  or  0  occurred  at  the  same  time 
to  different  groups.    Thus  the  provisional  design #  the 
"One  Group  Pretest-Posttest  Design" —duplicated  in  two 
organizations  can  be  symbolized  in  the  following  manner: 

Fall,  1970  Spring,  1971 

Experimental  Group  #1  0  X  0 

Experimental  Group  #2  0  X  0 

The  above  symbolism  reflects  the  fact  that  data 
(0)  is  to  be  gathered  from  an  organization  prior  to  the 
introduction  of  a  new  program  (X)  and  then  gathered  after 
the  program  is  over.    The  process  is  repeated  in  both 
organizations.    There  is  no  control  group? H  this  is 
reflected  by  the  absence  of  a  third  row  without  an  (X). 
Both  the  observation  (0)  and  programs  occur  simultaneously 
in  each  of  the  organizations.    Lack  of  a  series  of  dashes 
between  rows  or  an  R  at  the  end  indicates  that  no  compari- 
son is  to  be  made  between  groups. 

In  this  case,  the  (X)  refers  to  the  AI!A  training 
program.    The  0  indicates  a  measurement  of  the  impact  of 
(X)  on  the  SED's/LEA's  with  respect  to  selected  dependent 
variables  previously  described.    Since  the  training  in  all 
SED's/LEA's  was  generiilly  limited  to  the  top  management 
of  the  agency  or  a  particular  division,  not  offered  to  a 
random  sample  of  the  total  administrative  population,  an 
(R)  is  inappropriate. 


In  experimentation  the  control  group  is  a 
standardizing  device.    The  control  group  constitutes  a 
base  line  standaurd  against  which  to  measure  the  experi- 
mentally treated  group*    The  key  idea  is  that  the  experi 
mental  group,  which  is  chosen  from  the  same  universe  as 
the  control  group,  would  show  the  same  results  as  the 
control  group  had  it  not  been  treated  experimentally. 
See:    Juliem  Simon,  Basic  Research  Methods  in  Social 
Science  (New  York:    Random  House,  1969),  p.  ^25. 
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This  last  characteristic  deserves  further  elabora- 
tion.   Because  the  experimental  groups  were  not  randomly 
selected,  the  possibility  that  the  results  of  the  study 
can  be  generalized  to  other  groups  (or  the  same  group  at 
other  times)  is  reduced. 12    Random  selection^^  of  a 
population  sample  has  the  advantage  that  it  is  possible 
to  make  statements  about  the  characteristics  of  the  total 
population,  or  in  this  case  of  other  education  agencies. 

This  is  not  intended  as  criticism  of  the  choice 
process  used  in  this  evaluation,  but  this  lack  of  random- 
ness characterizes  most  field  studies  of  the  effects  of 
training  in  an  ongoing  organizational  setting.  Partici- 
pants select  themselves  into  the  experimental  groups  on 
the  basis  of  characteristics  extraneous  to  the  research 
problem;  examples  of  these  factors  are  hierarchical  rank, 
work  group,  useful  influence,  prior  experience,  etc. 

In  the  study  of  the  AJlA's  program,  the  research 
team  was  given  an  experimental  group  and  asked  to  evaluate 
the  impact  of  the  training  on  them  and  on  their  organiza- 
tions. 

The  self-selection  process''"'^  is,  however,  one  of 
many  factors  which  can  adversely  affect  the  strength  of 
a  research  design. 
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12 

See  Sir  Ronald  Fisher,  Design  of  Experiments 
(Edinburgh:    Oliver  6  Boyd,  195171 —    


13 

The  principle  of  randomisation  means  that  every 
member  of  a  population  Eas  an  equ  u  chcince  of  being 
selected.    Members  with  certain  distinguishing  characteris- 
tics— male  and  female.  Republican  and  Democrat,  extrovert 
and  introvert,  high  and  low  intelligence,  and  so  on  and 
on— will,  if  selected,  probably  be  counterbalanced  in  the 
long  run  by  the  selection  of  other  members  with  the 
"oppoLite"  quantity  or  quality  of  characteristics.  We 
might  say  that  this  is  a  practical  principle  that  indicates 
what  happens.    It  is  simply  a  statement  of  what  usually 
happens  when  random  procedures  are  used.    Ker linger,  op. 
cit. ,  p.  57. 

14 

W.  Richard  Scott,  "Field  Methods  in  the  Study  of 
Organi-sations,*  in  James  G.  March  (ed.).  Handbook  of 
Organizations  (Chicago:    Rand  McNally,  1955),  pp.  5bl-303. 
See  also: Egon  G.  Guba,  "Common  Sense  About  Experimental 
Design  and  Educational  Research,"  paper  read  at  a  faculty 
seminar.  School  of  Education,  New  York  University,  25  Feb. 
1963 . 
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Section  3:    Threats    to  Validity 

Validity  is  the  overall  concept  used  to  refer  to 
the  potential  accuracy  of  an  evaluation.     In  their  work 
on  quasi-^experimental  design,  Campbell  and  Stanley^^  have 
identified  fifteen  factors  which,  if  not  brought  under 
control  by  the  research  design,  may  complicate  efforts 
to  measure  training  effects.    They  are  called  "threats  to 
validity"  and  fall  into  two  general  categories?  internal 
and  external: 

Internal  Threats  to  Validity:    extraneous  variables 
which  may  have  a  significant  impact  on  the  experimental 
group  yet  go  unmeasured  by  research  instruments,  causing 
us  to  attribute  greater  effect  to  the  training  than  is 
warranted.    Internal  validity,  then,  answers  the  question: 
To  what  extent  can  we  clearly  assess  whether  the  training 
program  really  made  a  significant  difference  in  the  actions 
and  attitudes  of  SED/LEA  personnel  who  participated  in  it? 
Clarity  of  assessment  is  conditioned  by: 

1.  History;    events,  other  than  the  experimental 
treatment,  occurring  between  pre  and  post  test  and  thus 
providing  alternate  explanations  of  events. 

2.  Maturation:    processes  within  the  trainees 
or  their  agencies  producing  changes  as  a  function  of  the 
passage  of  time  per  se,  such  as  growth,  fatigue,  secular 
trends,  etc. 

3.  Instability:    unreliability  of  measures, 
fluctuations  in  sampling  persons  or  components,  autonomous 
instability  of  repeated  or  equivalent  measures.  (This 

is  the  only  threat  to  which  statistical  tests  of  signifi- 
cance are  relevant.) 

4.  Testing:    the  effect  of  taking  a  test  upon 
the  scores  of  a  second  testing.    The  effect  of  publication 
of  a  social  indicator  upon  subsequent  reading  of  that 
indicator. 

5.  Instrumentation :     in  which  changes  in  the 
calibration  of  a  measuring  instrument  or  changes  in  the 


^^Much  of  this  material  was  taken  directly  or 
adopted  from  the  Kirkhart  and  Tanner  report,  pp.  19-23. 

^^Campbell  and  Stanley,  o£.  cit . ,  pp.  5-6. 
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observers  or  scores  used  may  produce  changes  in  the 
obtained  measures. 

Regression  Artifacts:    pseudo-shifts  occur- 
ring when  persons  or  treatment  units  have  been  selected 
upon  the  basis  of  their  extreme  scores. 

7.    Selection;    biases  resulting  from  differential 
recruitment  ot  comparison  groups,  producing  different  mean 
levels  on  the  measure  of  effects. 

8-    Experimental  mortality:    the  differential 
loss  of  respondents  from  comparison  groups. 

9«  Selection-maturation  interaction ;  selection 
biases  resulting  in  differential  rates  of  ^'maturation"  or 
autonomous  change. 


External  Threats  to  Validity;    factors  which  impede 
our  ability  to  generalize  the  information  gained  from  one 
experimental  group  to  other  experimental  groups. 17  Here 
we  are  concerned  with  problems  like;     "To  which  other 
SED's/LEA's  can  the  information  gained  by  this  evaluation 
be  generalized?"    "All  SED's/LEA's?"    "Only  those  in  the 
same  region?"    "Merely  to  the  same  group  at  a  different 
time?"    "Or  can  it  be  generalized  at  all?"    The  major 
consequence  of  not  controlling  for  these  threats  is  to 
dramatically  reduce  or  even  make  impossible  the  extension 
of  findings  to  other  contexts.    These  threats  arel 

1.  Interaction  effects  of  testing:    the  effect 
of  a  pretest  in  increasing  or  decreasing  a  respondent's 
sensitivity  or  responsiveness  to  the  experimental  variable, 
thus  making  the  rc  suits  obtained  for  the  pretested  popula- 
tion unrepresentative  of  the  effects  of  the  experimental 
variable  for  the  unpretested  total  population  from  which 
respondents  were  drawn. 

2.  Interaction  of  selection  and  experimental 
treatment:    unrepresentative  resDonsiveness  of  the  experi- 
mental  population. 

3«    Reactive  effects  of  experimental  arrangements; 
artificiality,  conditions  that  render  the  experimental 


Glen  H.  Bracht  and  Gene  V.  Glass,  "The  External 
Validity  of  Experiments  in  Education  and  the  Social 
Sciences,"  (Research  Report  #3,  Laboratory  of  Educational 
Research,  University  of  Colorado,  Oct.,  1967). 
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setting  atypical  of  conditions  of  regular  application  of 
the  treatment:    "Hawthorne  Effects." 

4 .  Multiple- treatment  interference:  where 
multiple  treatments  are  jointly  applied,  producing  effects 
atypical  of  the  separate  application  of  the  treatments. 

5.  Irrelevant  responsiveness  of  measures:  all 
measures  are  complex,  and  all  include  irrelevant  components 
that  may  produce  apparent  effects. 

6.  Irrelevant  replicability  of  the  treatments: 
treatments  (experiments)  are  complex  and  replications  of 
them  may  fail  to  include  those  components  actually 
responsible  for  the  effects. 

Since  we  are  evaluating  the  effects  of  training  on 
a  nonrandom  sample  in  a  guasi**expGrimental  field  situation, 
it  will  not  be  possible  to  guarantee  that  our  findings  have 
generalizability  or  external  validity.    Therefore  we  must 
restrict  ourselves  to  the  nine  threats  to  internal  validity 
dejjailed  above  and  describe  what  kinds  of  strategies  are 
relevant  to  controlling  for  or  reducin<^  the  impact  of  these 
threats  to  our  research  data. 

Here  we  are  largely  on  our  own  as  we  are  aware  of 
no  available  document  which  syst»^L  atically  deals  with  this 
problem.    On  the  other  hand,  there  is  a  series  of  strategies 
implied  in  the  work  of  Campbell  and  Stanley  which  we  shall 
attempt  to  make  explicit. 

According  to  our  analysis,  the  nine  threats  to 
internal  validity  can  be  grouped  into  three  areas:  (1) 
factors  affected  by  the  presence  of  control  groups,  (2) 
factors  affected  by  the  manner  m  which  the  measure'^ent 
process  is  handled,  and  (3)  ^<  factor  which  cuts  across 
both  of  the  former  areas. 


Strategies  for  Minimizing  Threats  to  Validity 

Conditions  Related  to  Presence  of  Control  Groups 
Threats  Reduced  by: 

1.    History  Addition  of  one  or  more  control  or  non- 

treated  groups,  preferably  selected  on  a 
random  basis 

and/or 

utilization  of  data  collected  over  an 
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Threats  Reduced  by? 

extended  period  of  time.    If  only  the 
latter  strategy  is  used,  it  is  critical 
that  the  measurement  process  by  which  the 
data  were  generated  remains  the  same. 
^   (Cf.  analysis  of  threat  ff5  below.) 

2.    Flaturation       Addition  of  one  or  more  control  groups, 

preferably  selected  on  a  random  basis. 


4,    Testing  Addition  of  at  least  one  control  group 

which  is  not  pretested,  assuming  both  the 
experimental  and  control  group  are  selected 
randomly 

or 

addition  of  at  least  two  control  groups — 
one  is  pre  and  post  tested,  the  second  is 
post  tested  only.     (Assuming  random  selec- 
tion processes,  differences  between  the 
first  cind  second  control  group  is  related 
to  testing.) 


6. 

Regression 
Artifacts 

Avoidance  of  the  use  of  groups  which  are 
extreme,  either  high  cr  low,' in  relation 
to  the  general  population  as  determined 
by  some  measurement  device. 

7. 

Selection 

Random  selection  of  experimental  group (s) 

and 

plus 

9. 

Selection 
Maturation 
Interaction 

examination  of  recruitment,  selection  and 
turnover  figures  in  the  case  of  groups 
which  have  existed  over  time  prior  to  the 
experiment. 

*  * 

******* 

********************** 

Conditions  Related  to  the  Measurement  Process 


Threats  Reduced  by; 

3.    Instability      (Assuming  the  presence  of  at  least  one 

control  group*)  Statistical  analysis  and 
probability  theory  plus  careful  design  of 
measurement  factors. 


5.    Instrument        (Assuming  the  presence  of  at  least  one 

tat ion            control  group.)    Not  modifying  measurement 
instrument  during  the  evaluation  and 
through  analysis  of  the  comparability  of 
"comparable"  or  "equivalent"  measurement 
methods,  if  these  are  to  be  used. 
O  .   ^  

ERJC  ********  *********************** 
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General  Condition 

Throat  Reduced  by: 

8.    Experimental    Random  selection  process  if  group (s)  is 
Mortality        (are)  to  ho  temporary  plus  (in  the  case 
of  non temporary  groups)  careful  pre  and 
post  analysis  of  mortality  rates  in  the 
population  studied. 


Each  of  these  techniques  is  a  way  in  which  the 
threats  that  these  factors  pose  to  internal  validity  may 
be  minimized.    Failure  to  apply  these  reduction  strategies 
on  a  particular  threat  are  grounds  for  discounting,  or  at 
least  opening  to  serious  question,  a  singular  interpretation 
of  the  research  findings.    For  example,  if  an  autocratic 
superintendent  of  schools  was  replaced  by  one  who  was  more 
participative  during  the  course  of  the  evaluation  after 
the  training  program  but  before  the  final  tests,  any  credit 
that  the  training  could  be  given  for  improving  organiza- 
tional climate  would  have  to  be  shared  with  the  extraneous 
personnel  change. 

A  key  concern,  therefore,  is  to  ask  what  controls 
to  minimize  those  threats  are  present  in  the  AMA  provisional 
research  design.    Unfortunately,  the  simple  fact  is  that 
the  design  contained  virtually  none.    Without  being  able 
to  compare  apparent  changes  in  groups  which  peurticipated 
in  the  program  with  a  control  group  which  did  not,  no  real 
defense  may  be  raised  against  the  charge  that  our  findings 
did  not  reflect  AMA  training  at  all  but  rather  the  effects 
of  one  or  more  threats  listed  above* 

In  addition,  the  lack  of  any  provision  for  random 
selection  of  the  groups  raises  the  issue  of  self -selection. 
Did  participants  really  represent  the  SED/LEA  in  question? 
Even  under  optimal  conditions,  this  last  problem  is  almost 
impossible  to  control  in  evaluating  an  already  existing 
program. 

For  these  reasons  the  provisional  research  design 
was  determined  to  be  less  than  satisfactory  if  the  purposes 
for  which  the  evaluation  was  commissioned  were  to  be  served. 
If  the  early  design  had  been  utilized,  there  would  have 
been  at  least  six,  and  perhaps  nine,  equally  plausible, 
irrefutable  explanations  of  the  research  data.    This  is 
indicated  in  the  table  below. 
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Section  4:    First  Year's  Evaluation  (YD;  Final 
"  Research  Design^y 

Because  of  these  concerns,  a  control  group  was  added 
to  the  Yl  evaluation.    This  third  group  was  selected  on 
the  basis  of  interviews  with  top  administrators  in  each  of 
the  experimental  states.    Each  person  was  asked  to  name 
an  SED  which  was  most  like  his  own.    From  these  inquiries, 
three  states  were  nominated  and  one  was  selected.  The 
top  administrator  of  the  proposed  state  was  contacted  and 
he  agreed  to  permit  his  agency  to  participate  m  the  Yl 
evaluation  as  the  control  group.    A  part  of  his  rationale 
for  accepting  this  role  was  the  promise  that  his  group 
would  have  access  to  the  research  findings,  and  the 
possibility  that  present  cooperation  might  give  them 
access  to  similar  training  if  the  program  expanded. 

20 

The  "Non-i::iUivalent  Control  Group  Design"  which 
grew  out  of  these  circumstances  is  described  below:  ^-^ 

Fall,  1970  Spring,  1971 

0X0   

Experimental  SED  #1  5  g 


0X0 
Experimental  SED  #2  q  0" 


Because  the  designs  for  both  the  first  and  second  years' 
evaluation  are  similar,  a  discussion  of  the  first  year  b 


■•■^Kirkhart  and  Tanner,  c^'  cit .  /  P.  26. 

^^Campbell  and  Stanley,  o£.  cit . ,  pp.  47-50. 

''^The  bottom  row  in  each  case  represents  the  same 
control  group. 
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design  has  been  combined  with  that  of  the  second  year's 
design/  and  follows  that  section  of  the  report. 


Section  5:    Second  Year's  Evaluation  (Y2) ;  Promise 
and  Performance 

Factors  which  v;ere  taken  into  account  when  design- 
ing Y2  centered  around  the  use  of  control  groups; 

1.  Addition  of  Second  Control  SEP;    As  they  had 
hoped  when  agreeing  to  participating  in  Yl^  the  control 
state  did  becOTie  involved  in  AI!A  training  .  .  .  which  began 
in  the  middle  of  Y2.     If  Y2  was  to  retain  the  important 
contribution  that  the  control  state  mechanism  made  to  the 
interpretation  of  research  findings,  it  was  necessary  to 
add  a  second  SED  which  was  not  exposed  to  A^!A  training 

for  comparison  to  the  two  .  .  .  and  now  three  .  .  .  states 
who  were.    You  may  remember  from  a  previous  section  that 
the  original  control  state  V7as  chosen  after  interviews  with 
top  administrators  in  the  tv/o  experimental  states  .  .  .  and 
that  it  had  had  the  highest  consensual  rating  of  the  three 
nominated  by  them.    To  obtain  the  second  control  state  for 
Y2/  the  research  team  approached  the  state  with  the  next 
highest  rating.    Fortunately,  they  also  agreed  to  partici- 
pate as  a  control  state.    Their  expressed  reasons  for 
signing  on  were  similar  to  those  of  the  first  control 
state:    access  to  research  findings  and  the  possibility  of 
receiving  training  if  the  program  expanded  a  second  time. 

The  Y2  evaluation  on  the  SED  level  thus  becomes 
tri- level:    two  experimental  states  whose  AMA  training 
ended  over  a  year  ago,  one  experimental/control  state  which 
has  just  completed  its  training  program  after  being  the 
control  state  during  Yl,  and  a  control  state  with  no  train- 
ing experience.    These  multivariate  comparisons  will  make 
the  Y2  research  design  even  stronger  than  the  one  used  in  Yl. 

2.  Addition  of  Tv;o  Control  LEA' s:    Y2  has  made 
provision  for  comparing  data  gathered  from  the  experimental 
lea's  with  an  appropriate  control  LEA.    Without  such  a 
basis  for  comparison,  the  validity  of  these  results  would 
have  been  as  uncertain  as  the  validity  of  SED  data  would 
have  been  without  their  control  group(s).    Our  LEA  evalua- 
tion under  these  circumstances  would  have  been  unable  to 
refute  the  r.rgument  that  the  apparent  effects  had  nothing 

to  do  with  training  but  were  the  results  of  history,  matura- 
tion, testing,  instrumentation,  or  instability.  There 
would  have  been  no  untreated  group  to  provide  baseline  data 
for  comparative  assessment  of  the  kinds  and  degrees  of 
change  alleged  to  have  emerged  from  AMA  training. 
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Realizing  this  threat  to  validity,  the  Y2  research 
team  added  one  control  LEA  to  the  two  experimental  LEA's 
in  each  state,  or  two  control  LEA's  in  total.  The  recruit' 
ment  process  was  a  slight  variation  of  that  used  on  the 
state  level.  We  asked  the  chief  planning  officer  in  each 
experimental  SED  to  suggest  an  LEA  in  his  state  which  was 
most  similar  to  the  two  experimental  LEA's.  The  proposed 
LEA  controls '.rare  contacted  and  each  agreed  to  participate. 


Section  6;    Macro  Yl  S  Y2  Research  Designs;    SEP  S  LEA  Levels 

Aside  from  the  extension  of  the  tine  frames  into  a 
second  year,  the  addition  of  these  control  groups  is  the 
major  change  made  by  Y2  to  the  basic  Yl  design.    The  symbols 
used  to  illustrate  this  design  are  as  follows: 


S 
Y 
M 
B 
0 
L 
S 

U 

s 

E 
D 


ESED  #1  =  experimental  state  education  department  #1 
ESED  ^2  =  experimental  state  education  department  #2 
CSED  #1  =  control  state  education  department  #1 

<also  used  in  Yl) 
CSED  #2  =  control  state  education  department  #2 

(added  for  Y2) 


ELBA  #1  =  experimental  local  education  agency  #1 

(same  state  as  ESED  #1) 
ELBA  ^1^2  =  experimental  local  education  agency  #2 

(same  state  as  ESED  #1) 
CLEA  #1  =  control  location  education  agency  #1 

(same  state  as  ESED  #1) 

ELEA  #3  =  experimental  local  education  agency  #3 

(same  state  as  ESED  #2) 
ELEA  #4  =  experimental  local  education  agency  #4 

(same  state  as  ESED  #2) 
CLEA  #2  =  control  local  education  agency  if 2 

(same  state  as  ESED  #2) 
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I.     SED  LEVEL 
ESED  #1 


CSED  ^il 
ESED  i\l 


CSED  #2 


Yl 


Y2 


Pall, 
1970 


0 
0 


Spring,  Frsll, 
1971  1971 


0 
0 


0 
0 
0 


Soring, 
1972 

 iPT  


0 

0 
0 
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tC] 


[D] 


[E] 


tP] 


[G] 


IH] 


ESED 

«2 

n 

X 

0 

0 

0 

CSED 

#1 

0 

0 

0 

X  0 

ESED 

#2 

0 

X 

0 

0 

0 

CSED 

n 

0 

0 

.EA  LEVEL 

ELEA 

#1 

0 

X 

0 

0 

0 

CLEA 

#1 

0 

0 

ELEA 

n 

0 

X 

0 

0 

0 

CLEA 

n 

0 

0 

ELEA 

#3 

0 

X 

0 

0 

0 

CLEA 

#2 

0 

0 

ELEA 

#4 

0 

X 

0 

0 

0 

CLEA  #2 


Explanation 

0  =  data  gathered  at  the  time  indicated  (Tl) 
.    thru  (T4> 

X  =  AMA  training  program 

Series  of  dashes  {  )  between  rows  =  a  nonrandom 

assignment  of  subjects  to  experimental 
groups 

[A]  thru  iH]  =  paired  comparisons  of  experimental 
and  control  groups 


I.    SEP  Level 

lA]  ^  Data  was  gathered  from  ESED  #1  (at  Tl)  prior  to  the 
introduction  of  AIIA  training  (X)  and  then  gathered 
three  times  after  training  was  completed  (T2-T4), 
l^Thile  data  was  gathered  from  CSED  #1  simultaneously, 
it  entered  AHA  training  subsequent  to  T3. 


[B]  =  Data  was  gathered  from  ESED  #1  (at  Tl)  prior  to  the 
introduction  of  AMA  training  (X)  and  then  gathered 
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three  times  after  training  was  completed  (T2-T4) . 
CSED  #2  was  added  as  a  control  group  for  ^2  due  to 
CSED  ftl  entering  AI4A  training  and  underwent  data 
collection  only  twice  (T3  &  T4). 

(Cl  =  Data  was  gathered  from  ESED  n  (at  Tl)  prior  to  the 
introduction  of  AMA  training  (X)  and  then  gathered 
three  times  after  training  was  completed  (T2-T4) . 
mile  data  was  gathered  from  CSED  #1  simultaneously, 
it  entered  AMA  training  subsequent  to  T3. 

[Dl  =  Data  was  gathered  from  ESED  #2  (at  Tl)  prior  to  the 
introduction  of  AMA  training  (X)  and  then  gathered 
three  times  after  training  was  completed  (T2-T4) . 
CSED  #2  was  added  as  a  control  group  for  ¥2  due  to 
C^tD  ffl  entering  AMA  training  and  underwent  data 
collection  only  twice  (T3  &  T4) . 


II.    LEA  Level 

[El  =  Data  was  gathered  from  ELEA_J1  (at  Tl)  prior  to  the 
introduction  of  AMA  training  Tx)  and  then  gathered 
three  times  after  training  was  completed  (T2-T4) . 
For  Y2,  CLEA  #1  was  added  as  the  control  group  for 
ELBA'S  ftl  &  2  and  underwent  data  collection  only 
twice  (T3  &  T4) . 

[F]  =  Data  was  gathered  fro*a  ELBA  #2  (at  Tl)  prior  to  the 
introduction  of  AMA  training  (X)  and  then  gathered 
three  times  after  training  was  completed  (T2-T4) . 
For  Y2  CLEA  #1  was  added  as  the  control  group  for 
ELBA'S  4l  &  #2  and  underwent  data  collection  only 
twice  (T3  &  T4) . 

[Gl  =  Data  was  collected  from  ELBA  #3  (at  Tl)  prior  to  the 
introduction  of  MV"  training  (X)  and  then  gathered 
three  times  after  training  was  completed  (t2'T4) . 
For  Y2,  CLEA  #2  was  added  as  the  control  group  for 
ELBA'S  #3  &  #4  and  underwent  data  collection  only 
twice  (T3  &  T4) . 

[HI  =  Data  was  gathered  from  ELBA  ft 4  (at  Tl)  prior  to  the 
introduction  of  AI4A  training  (X)  and  then  gathered 
three  times  after  training  was  completed  (T2-T4) . 
For  Y2  CLEA  ff2  was  added  as  the  control  group  for 
ELBA'S  IT&IR  and  underwent  data  collection  only 
twice  (T3  &  T4) . 
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Section  7;    Effectiveness  of  the  Overall  Research  Design 

T^ith  the  exception  of  the  lack  of  random  selection, 
this  design  approximates  a  true  experimental  model  that 
minimizes  threats  to  internal  validity.    But  the  lack  of 
random  selection  cannot  be  dismissed  as  having  no  effect 
on  our  findings.    While  additional  controls  have  been 
added,  this  lack  of  random  selection  still  makes  a  differ- 
ence between  the  Y1/Y2  design  and  a  true  experiment  in 
three  areas:     (1)  selection-maturation  interaction,  (2) 
regression  artifacts,  and  (3)  experimental  mortality. 

1.    Selection-^Maturation  Interaction  differences 
are  largely  contined  to  the  recurring  problem  of  matched 
groups  vs.  remdomly  selected  groups.    We  have  already 
conceded  the  impossibility  of  random  selection  due  the 
nature  of  the  training  program  itself. 

As  far  as  equivalence  (matching)  is  concerned,  Y2 
contains  tx^o  partial  controls.    First  it  should  be  noted 
that  the  addition  of  even  an  unmatched  or  nonequiyalent 
group  greatly  reduces  this  threat  to  validity  as  found  in 
the  "One  Group  Pretest/Posttest  Design"  proposed  in  the 
AMA's  provisional  design.    The  impact  that  each  training 
program  has  on  the  experimental  groups  is  thereby  made 
"equivalent"  in  relation  to  the  control  group (s)  which  did 
not  receive  training.    This  division  allows  us  to  avoid  a 
number  of  questionable  assumptions  about  the  extent  to 
which  the  experimental  groups  were  similar  and  reduces  the 
problem  of  defending  the  comparability  between  the  experi- 
mental and  control  groups.     Second,  the  addition  of  a 
second  control  state  in  Y2  is  a  further  refinement  toward 
establishing  some  limited  evidence  of  presampling  matching. 

Of  course  more  could  have  been  done.    The  equiva- 
lence of  groups  can  be  determined  by  using  the  means  and 
standard  deviations  of  the  pretests  as  well  as  by  checking 
their  distributions.    For  this  purpose,  t  tests  and  F  tests 
are  acceptable.     If  all  items  prove  out  via  these  methods, 
one  can  proceed  with  instrument  administration  with  at 
least  scxne  assurance  that  there  is  clear  evidence  of  the 
extent  the  groups  were  or  were  not  matched  before  the 
experiment  began. 22 

But  we  must  emphasize  that  these  techniques  or  any 
other  of  the  frequently  used  means  of  making  adjustments 
(analysis  of  covariance,  using  difference  scores)  are  not 
as  effective  as  randomization  for  controlling  the  selection- 
maturation  interrelation  threat  to  internal  validity.  As 


Kerlinger,  0£.  cit. ,    p.  43. 
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Frederic  Lord  has  written:         .  .  there  simply  is  no 
logical  or  statistical  procedure  that  can  be  counted  on 
to  make  proper  allowance  for  uncontrolled  preexisting 
differences  in  groups. "23 

2.  The  factor  of  regression  artifacts  has  been 
brota^t  under  control  by  the  greater  longitudinal  perspective 
afforded  by  the  addition  of  another  control  group.  The 
original  regression  problem  was  the  difficulty  of  separating 
real  from  pseudo  changes  given  the  nonrandom  selection  of 
the  experimental  groups.    However^  since  we  are  now  able 

to  plot  out  4  sets  of  results  on  a  2  year  time  frame 
instead  of  2  sets  of  results  on  a  1  year  time  frame, 
we  have  an  improved  sense  of  the  true  direction  and  levels 
of  change  brought  about  by  AMA  training.    The  1-2  year 
comparison  with  2  control  groups  further  lessens  the 
effects  of  regression  artifacts. 

3.  Finally,  it  is  possible  to  effect  control  over 
the  variable  of  Mortality  through  an  examination  of  the 
extent  to  which  ditferential  mortality  occurred  betv;een 
the  experimental  and  control  groups  when  the  posttest  data 
was  gathered. 

To  summarize,  the  control  effects  of  the  macro 
design  used  for  Yl  and  y2  on  the  SED  and  LEA  levels  are  as 
follows: 


Note:    Plus  (+)  -  control  over  the  threat  to  validity 


23 

Frederic  ?t.  Lord,  "A  Paradox  in  the  Interpre- 
tation of  Group  Comparisons,"  Psychological  Bulletin 
(1967),  p.  305.  
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Section  8:    Further  Strengthening  Program  Assessment: 
Improvements  Made  by  Y2 

1.    Pretest  of  Instruments;    There  are  several 
methods  to  provide  evidence  that  instruments  are  reliable ^ 
i.e.,  that  the  information  they  produce  will  tend  to 
remain  stable  over  time,  all  other  things  being  equal. 
If  we  test  the  same  group  again  and  again  with  the  same 
or  similar  instruments  will  we  obtain  the  same  or  similar 
results?    As  such,  reliability  is  an  index  of  random 
variance  in  the  results  of  a  study  and  tries  to  minimize 
its  impact. 

While  reliability  is  not  the  most  important 
characteristic  of  "good"  analysis,  it  is  still  essential. 
High  reliability  does  not  guarantee  accurate  results  and 
logical  conclusions,  but  without  it  we  can  never  be  sure 
if  our  apparently  logical  conclusions  are  based  on  accurate 
results.    Its  absence  leaves  the  evaluator  open  to  the 
charge  that  his  results  were  biased  by  instruments  too 
insensitive  to  measure  low  level  changes  or  too  crude  to 
be  consistent  over  time. 

The  reliability  of  Yl  was  established  by  one  method. 
The  questionnaire  was  based  on  another  instrviment  of  proven 
relieJ^ility  which  had  already  been  used  in  simileir 
circumstances . 

Another  method  was  added  for  Y2.    Using  essentially 
the  same  Yl  questionnaire  (with  a  few  revisions) ,  a  gre^ 
test24  was  run  on  the  Y2  questionnaire.    The  pretest  or 
Instruments  is  another  way  to  certify  test  reliability. 
Mistakes  at  the  analysis  stage  can  be  remedied  at  almost 
no  cost  prior  to  being  put  into  print  or  administered.^^ 
But  just  deciding  to  pretest  is  simple i  deciding  how  to 
pretest  is  not  so  simple. 26    The  Phi  Delta  Kappa  National 


Our  instruments  consisted  of  scaled  response 
questionnaires  and  semi-directed  interviews .  Pretesting 
reters  more  specifically  to  the  questionnaires. 

^^Lee  J.  Cronbach,  Essentials  of  Psychological 
Testing  (New  York:    Harper  &  Row,  1949) . 

^^Insofar     as  clarity  of  instructions  was  concerned, 
interview  reliability  was  pretested  on  a  small  sample.  It 
was  determined  that  questions  could  be  easily  understood 
and  related  to  the  respondents.    In  addition,  care  was  taken 
to  administer  them  under  standard,  well  controlled,  and 
similar  conditions  to  lesson  error  variance  due  to  situa- 
tional  ambiguity. 

ERLC 


56 


Study  Committee  on  Evaluation  points  out  that  there  are 
many  different  routes  that  an  evaluator  can  take: 

Determination  of  reliability  may 
be  accomplished  by  such  techniques 
as  split-half  correlation  (yielding 
a  measure  of  internal  consistency) / 
comparable  forms  correlation  (yield- 
ing a  coefficient  of  equivalence) , 
correlation  of  the  seuae  for  at  dif- 
ferent times  (yielding  a  coefficient 
of  stability) ,  or  correlation  of 
comparable  forms  at  different  times 
(yielding  a  coefficient  of  equiva- 
lence and  stability) . 2' 

On  the  basis  of  our  inquiries,  we  decided  to  run  a 
two-way  analysis  of  variance  as  well  as  a  Spearman's  Ro  on 
the  data.    Our  pretest  group  consisted  of  a  group  of  educa- 
tional administrators  in  the  Office  of  Residential  Life  at 
Syracuse  University.    It  was  composed  of  central  staff 
members  as  well  as  residence  hall  directors  and  resident 
advisors  in  the  field  (N-34) .    Each  was  given  the  same 
questionnaire  on  two  consecutive  Mondays  and  asked  to 
return  it  in  a  sealed  envelope  on  the  same  day.  Question- 
naires were  coded  to  facilitate  paired  comparisons.  The 
variance  on  week-to-week  responses  per  respondent  were 
within  acceptable  limits.    The  Spearman's  Rg  showed  high 
correlations  with  the  lowest  being  .41  but  most  being  in 
the  .70+  range.    With  this  result,  it  was  decided  that 
questionnaire  reliability  was  satisfactory. 

This  y2  pretest  was  an  ex  post  facto  check  on  those 
items  from  the  Yl  questionnaire  coHtTnued  on  the  ¥2  ques- 
tionnaire.   It  also  provided  evidence  of  the  reliability  of 
those  new  items  designed  specifically  for  the  Y2  question- 
naire.   We  could,  therefore,  proceed  with  the  instrument  as 
written. 

2.    Analysis  of  Environmental  Impact;    Mi  important 
part  of  the  evaluation  of  any  training  program  is  an  assess- 
ment of  the  effects  of  return  to  the  work  environment  on 
the  lessons  learned  and  the  plans  produced  by  the  trainees.* 
To  accomplish  this  the  analysis  should  not  take  Place 
during  or  immediately  before/after  training.    It  should 


^^Phi  Delta  Kappa  National  Study  Committee  on  Evalua- 
tion, Lducational  Evaluation  and  Decision  Making  (Blooming- 
ton,  Indiana;    Phi  Delta  Kappa,  inc.,  lavl) ,  p.  28. 

^Saniel  Katz  and  Robert  Kahn,  The  Social  Psychology 
of  Organizations  (New  York:    John  Wiley,  1966),  pp.  390-391. 
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await  reassertion  of  the  organizational  environment  in 
which  the  trainees  will  have  an  opportunity  to  act  out 
their  training. 

...  in  this  milieU/  many  of  the 
determinants  of  organizational 
behavior^  which  are  absent  in  off- 
site  training  situations ^  have  full 
sway.    The  role  sets^  group  and 
orgemizational  norms,  constraints 
which  grow  out  of  the  absence  or 
presence  of  technology,  and  the 
influence  of  the  organizational 
environment  are  the  force  field 
against  which  training  efforts  are 
ultimately  applied.    If  the  train-- 
ing  effort  is  to  have  the  effect  of 
changing  patterns  of  behavior  in  the 
organization,  this  set  of  factors 
must,  in  some  v;ays,  change.  29 

The  effect    of  environmental,  determinants  is  always 
basic  and  prior  to  the  others.    A  plan  or  program  must 
be  assessed,  at  least  in  part,  according  to  its  ability 
to  withstand  pressures  in  the  environment.    A  course  of 
action  which  may  seem  eminently  reasonable  in  the  classroom 
might  not  seem  so  great  to  the  folks  back  home  or  to  the 
trainees  themselves  once  they  return  to  the  realities  of 
administrative  life. 

But  since  considerable  overlap  already  existed 
between  the  training  and  evaluation  cycles  during  Yl,  the 
extent  to  which  such  a  study  was  possible  was  limited.  As 
Y2  began  several  months  after  the  training  (at  least  in 
the  experimental  states) ,  and  will  end  after  another  year 
has  passed,  it  had  a  good  opportunity  to  assess  the 
extended  effects  of  AIIA  training  on  the  respective 
organizational  environments. 

3.    Longitudinal  Perspective:    Another  spinoff  of 
the  proximity  of  the  training  and  evaluation  cycles  was  the 
limitation  placed  on  examining  training  impacts  over  time, 
especially  after  the  training  was  completed.    Such  a  long 
view  is  known  as  the  "time  series,"  "historical,"  or  as  in 


Larry  Kirkhart  and  W.  Lynn  Tanner,  "Evaluation 
for  Center  for  Planning  and  Development  of  the  American 
Management  Association,"  Report  submitted  to  the  American 
Management  Association  and  the  United  States  Office  of 
Education,  Syracuse  University  (Oct.  1971),  p.  30. 
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this  study/  the  "longitudinal**  method.    Its  main  advantage 
is  that  it  helps  us  to  get  some  idea  of  the  ability  of 
"changed"  attitudes  and  actions  to  remain  relatively 
constant  over  the  long  haul.    Changes  measured  too  close 
to  the  time  of  training  may  quickly  dissipate  or  may 
increase  after  a  few  days,  months,  or  years.    An  absence 
of  change  in  one  vari£jt)le  at  one  point  in  time  may  take 
on  a  positive  or  negative  direction  at  another.    With  a 
greater  longitudinal  perspective  we  can  strengthen  our 
confidence  that  training  was  indeed  adequate  preparation 
for  organizational  life. 

But  a  longitudinal  perspective  is  not  an  absolute 
good — extending  the  time  frame  does  not  always  bring 
corresponding  increases  in  confidence  in  reseeurch  results. 
Evaluating  too  long  after  the  conclusion  also  increases 
the  impact  of  the  internal  threats  to  validity  outlined 
previously  on  the  effects  of  training.    It  also  decreases 
the  relaveuice  of  the  original  data  to  data  gathered  much 
later.    After  all,  information  relating  to  schools  in  1945 
is  apt  to  be  misleading  today  due  to  external  factors, 
changes  in  data  reporting  and  collection  procedures,  etc. 
Or  to  cite  Brassier  and  Tumin's  exemiple: 

If  ...  we  wished  to  measure  the 
capacity  of  a  social  studies  curricu- 
lum to  modify  racial  attitudes  in  a 
class  that  entered  in  1964  and  gradu- 
ated in  1968,  how  could  we  protect 
the  inquiry  from  the  "contamination" 
of  intrusive  events  in  the  era  of 
Lyndon  Johnson,  Stokley  Carmichael, 
and  the  death  of  Martin  Luther  King?-^" 

Finally,  the  likelihood  that  goals  or  procedures  developed 
during  training  will  remain  stable  over  more  than  a  few 
years  is  remote  givea  the  fact  that  "education  for  change" 
and  continuous  emergence  of  new  programs  is  taken  as  a 
natural,  desirable  condition  of  the  system.    This  compli- 
cates the  evaluation  process  considerably. 

Therefore  our  decision  criterion  was  to  take  as 
long  a  view  as  possible  without  going  so  far  that  our 
conclusions  would  be  severely  threatened  by  the  inevitable 
effects  of  the  passage  of  time.    Nor  did  we  want  to  lose 
continuity  with  the  Yl  research  team  or  personnel  from  the 
experimental  groups. 


^^Mervin  Bressler  and  Melvin  Tumin,  Evaluation  of 
the  Effectiveness  of  Educational  Systems,  Vol.  1  (Prince- 
ton, N«J. :    Princeton  University  Press,  1969),  B9. 
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It  was  decided  that,  given  these  considerations,  an 
evaluation  covering  an  entire  year  after  training  was 
conpleted  would  be  optimal,    H  longitudinal  perspective 
was  gained  using  the  baseline  data  of  Yl,    The  two  yec ts 
together  can  be  valuable  to  those  undertaking  future 
evaluations  of  this  or  similar  programs. 

4,  Inclusion  of  LEA  Datat    The  Yl  research 
design  included  an  analysis  of  effects  of  AMA  train.  Aig  on 
the  four  experimental  local  education  agencies  which 
participated.    Data  was  collected  from  them  in  a  manner 
similar  to  that  used  in  the  SED's*    Yl  promised  that  these 
results  would  be  available  when  the  doctoral  dissertation 
of  one  of  the  members  of  the  resecurch  team  was  completed. 

Using  the  basic  Yl  design  and  instrumentation,  Y2 
has  continued  analysis  on  this  level.    The  Y2  evaluation 
report  will  contain  data  gathered  during  two  years  and 
appropriate  conclusions  drawn  from  them. 

5.  Addition  of  Organizational  Output  Data:  The 
basic  premise  of  Yl  was  that  Its  proper  focus  was  the 
measurement  of  attitudinal  change  as  a  precondition  for 
organizational  changes.    Unless  participant  attitudes  are 
changed  by  training /  its  effects  are  likely  to  be  temporary. 

In  addition,  since  the  ending  dates  for  the  evalua- 
tion and  training  were  almost  coterminous,  an  assessment 
of  the  structural  and  behavioral  impacts  of  the  MVi  program 
would  have  been  premature.    No  organization  is  able  to 
implement  the  results  of  a  training  program  in  a  few  weeks. 

The  attitudinal  measures  i^sed  in  Yl  have  been  only 
slightly  revised  for  Y2.    Whatever  revisions  which  have 
been  made  have  centered  on  the  added  dimension  of  actual 
organizational  output  as  a  result  of  training.  These 
concerns  of  y2  include  analyzing  the  plans  produced 
subsequent  to  AJIA  training,  progress  toward  educational 
objectives  recommended  therein,  ani  other  elements  of  the 
planning  process.    It  is  through  such  an  examination  that 
the  impact  of  the  progrcun  on  participant  and  organizational 
attitude  and  actions  can  best  be  assessed. 


Kirkhart  and  Tanner,  0£.  cit. ,  p.  40n. 
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Strategies  for  Strengthening  Overall  Evaluation 


Target  Areas 

1.    Pretest  of 
Instruments 


Improvement  made  by; 

Y2  questionnaire  was  pretested  on  a 
sample  drawn  from  the  Syracuse  Univer* 
sity  Office  of  Residential  Life  and 
analyzed  using  tv/o-way  analysis  of 
variance  and  Spearman's  Rs-    In  so  far 
as  clarity  was  concerned^  the  interview 
questions  were  protested  on  a  small 
sample  of  persons  known  to  the  research 
team. 


2.    Analysis  of  Analysis  of  impacts  of  training  one 

Environmental       year  after  the  training  has  ended 
impact 

after  participants  have  returned  to 
their  education  agencies. 


3.    Longitudinal         Given  the  necessity  for  keeping  the 
Perspective  research  team  intact/  reducing  the 

mortality  rate  of  the  training  group , 
and  minimizing  the  effects  of  external 
variables/  the  time  frames  for  evalua- 
tion were  extended  by  one  year  by  Y2. 


4.    Inclusion  of         Evaluation  report  contains  data  and 
LEA  Data  conclusions  for  all  participating 

lea's  in  the  AMA  training  program  for 
both  evaluation  years  (Yl  &  Y2) . 


5.    Organizational     While  the  attitudinal  measurements 
Output  Data  used  in  Yl  have  been  only  slightly 

revised  for  Y2/  additional  instruments 
were  added.  This  includes  an  analysis 
of  the  plans  produced/  progress  toward 
action  plan/  continuing  and  specific 
objectives/  money  made  availcO^le  for 
use  by  educational  planners/  etc.  (cf . 
Appendix) . 
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CHAPTER  THREE 


RESEARCH  METHODOLOGY 


A  fundamental  problem  of  evaluation  is  to  define 
what  is  to  be  evaluated  and  then  how.    This  task  has  been 
made  somewhat  easier  in  the  Y2  evaluation  because  it  is 
related  to  the  Yl  evaluation.    As  is  discussed  above,  the 
overall  evaluation  is  a  two*year  longitudinal  study. 


Section  1;    Research  Methodology  -  Yl 

The  research  methodology  employed  in  Yl  was 
developed  after  a  careful  analysis  of  the  training  design 
employed  by  the  AHA  as  well  as  of  the  way  it  was  imple* 
mented.    As  Chapter  One  reported,  the  first  year's  research 
team  developed  the  conceptual  linkages  of  (1)  individual 
awareness/knowledge,  and  (2)  role  relations  and  group 
standards  through  which  chanqe— -if  it  was  to  occur  when 
participants  returned  to  the  organizational  setting*«must 
be  transmitted.    These  linkages  were  operationalized  by 
grouping  around  them  the  variables  in  the  study.  The 
two  categories  corresponded  to  Likert's  concepts  of  causal 
and  intervening  variables.^    The  Y2  evaluation  has  utilized 
the  same  two  general  categories,  which  are  presented  in 
Chapters  Pour  and  Five. 

Data  were  collected  by  means  of  three  techniques: 
structured  questionnaires,  semi*structured  interviews, 
and  observations  of  training—these  instruments  sought  to 
measure  the  effect  of  training  on  attitudes. 

The  first  year's  evaluation  did  not  try  to  measure 
organizational  output  regarding  planning.    As  the  Yl  team 
pointed  out,  while  a  set  of  documents  (action  plans) 
was  produced  for  each  SED  and  LEA  during  ^:he  training 
process,  the  documents       .  .  represented  intentions,  not 
necessarily  processes  and  policies  which  have  had  the 
opportunity  to  be  implemented  and  affect  organizational 
output. 2 


Rensis  Likert,  The  Human  Organization  (New  York, 
1967),  pp.  28-29. 

2 

Larry  Kirkhart  and  W.  Lynn  Tanner,  "Evaluation 
for  Center  for  Planning  and  Development  of  the  American 
Management  Association''  (Syracuse,  New  York:  Syracuse 
Q  University,  October,  1971),  p.  40. 
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Soction  2;    Research  Methodology  -  Y2 


The  present  evaluation's  primary  focus  has  been  to 
measure  the  impact  jf  the  AMA's  training  program,  over 
time  as  well  as  in  the  context  of  the  organizational 
environment:    how  the  program  affected  attitudes,  actions, 
and  organizational  output  in  the  experimental  organizations. 
To  l^is  end  we  have  maintained  and  expanded  the  two  cate- 
gories originally  developed  by  the  Yl  research  team  and 
have  ac'.ded  a  third  concerning  the  end-result  output  of 
these  organizations.    In  each  category  we  have  used  the 
data  developed  through  the  evaluative  techniqiies  discussed 
below. 

As  in  the  Yl  evaluation,  we  have  conceptualized 
the  research  variables  of  this  project  in  terms  of  Likert's 
causal  and  intervening  variables.    We  have  also,  of  course, 
added  his  third  category,  end-result  variables.  Likert 
defines  tliese  categories  as  follows: 

The  "causal  variables  are  independent 
variables  which  determine  the  course 
of  developments  within  an  organization 
and  the  results  achieved  by  the  organi- 
zation.   These  causal  variables  include 
only  those  independent  variables  v.hich 
can  be  altered  or  changed  by  the  organi- 
zation and  ita  management. ^ 

The  "intervening"  veiriables  reflect  the 
internal  state  ind  health  of  the  organi- 
zation, e.g.,  the  loyalties,  attitudes, 
motivations,  performeince  goals,  an'J 
perceptions  of  all  members  and  their 
collective  capacity  for  effective  in- 
teraction, communication,  and  decision 
making . ^ 

The  "end-result"  variables  are  the 
dependent  variables  which  reflect  the  5 
achievements  of  the  organization.  .  .  . 

Grouping  the  research  variables  into  these  broad 
categories  establishes  a  linkage  whi'  '  enables  us  to 
assess  more  "iccurately  the  impact  of  th«3  training  on  the 
organizations  in  terms  of  attitudes,  actions,  and  results, 
as  well  as  to  examine  the  interrelation  among  these  variables. 


*Likert,  0£.  cit. ,  p.  40. 

ERIC 


63 


Section  3:    Measurement  of  the  Impctct  of  Training 

Three  techniques  were  employed  to  gather  data  during 
the  second  year  evaluation:     scaled  response  questionnaires 
(SRQ),  open-ended  interviews,  and  an  analysis  of  organiza- 
tional documents.    All  three  techniques  were  applied  to  the 
two  original  experimental  state  educational  agencies  and 
the  data  was  collected  on  site.    The  SRO's  and  open-ended 
interviews  were  also  used- in  Control  SED  f?l  while  only  the 
SRQ»s  were  completed  in  Control  SED  #2.    All  the  local 
educational  agencies  completed  the  SRQ's, 


Section  4;    The  Use  of  Scaled-Response  Questionnaires 

Questionnaires  were  utilized  in  this  research  because 
they  are  easy  to  administer  and  are  pre-coded.    They  also 
produce  large  amounts  of  data  while  requiring  a  relatively 
short  period  of  intervention  in  the  organization's  processes. 
The  primary  disadvantage  of  structured  questions  is  that 
they  sacrifice  much  of  the  color  and  intensity  of  the 
respondent's  answer.^    Questionnaires  also  may  induce  a 
compliance  process  on  the  respondent's  part— people  say 
what  they  think  they  should  say.    This  is  termed  the  prob- 
lem of  reactive  measurement  and  is  a  significant  issue  in 
social  science  research.'' 

The  problem  of  interpretation  of  the  intensity  of 
the  respondent's  opinions  is  partially  overcome  by  using  a 
rating  scale  which  allows  the  respondent  to  indicate  the 
direction  and  intensity  of  his  feelings.    The  questionnaire 
emp-^oyed  in  the  present  research  used  a  seven-point  scale 
discussed  below.    To  control  and  check  for  the  problem  of 
reactive  measurement,  we  consciously  attempted  '  )  allow 
major  areas  of  overlap  in  the  questionnaire  and  open-ended- 
interviews . 


Section  5;    The  Design  of  the  Questionnaire 

Because  this  is  a  longitudinal  study,  the  question- 
naire aesign  wc  used  was  tuat  of  the  Yl  instrument.    As  we 


^Charles  H.  Backs trom  and  Gerald  D.  Hursh,  Survey 
Research  (Minneapolis,  flinn.,  1963)  •  p.  75. 

^Kirkhart  and  Tr.nner,  0£.  cit. ,  p.  3b. 

®The  entire  questionnaire  as  administered  in  the 
State  Education  Departments  and  the  Local  Education  Agencies 
is  located  in  Appendix  D  ot*  this  report. 
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stressed  above,  this  choice  was  partly  a  recognition  of 
the  critical  importance  of  using  the  same  instruments  and 
procedures  to  insure  the  reliability  and  relatedness  of 
the  data  drawn  from  both  years  of  the  study. 

The  Yl  questionnaire  was  designed  around  two  broad 
categories  related  to  organizational  planning  and  the 
managerial  environment.    After  a  factor  analysis  of  these 
items,  the  Y2  research  team  selected  28. items  which  were 
highly  correlated  around  these  categories.    In  addition, 
to  meet  the  expanded  needs  of  the  Y2  effort  ^.he  research 
team  addec  29  new  items. 

The  questionnaire  items  clustered  around  three 
categories;  the  first  is  related  to  goals  of  the  WiA  pro- 
gram which  involved  organizational  planning.    The  second 
set  of  items  reflect  WtA's  process  objectives:    what  they 
wished  to  induce  in  tha  management  environment  of  the 
experimental  organizations.    The  third  category  seeks  to 
ioolate  end-result  variables. 

Organizational  Planning  Process 

The  following  items  have  been  continued  from  the 
Yl  questionnaire. 

1.  The  kinds  of  things  I  am  doing  will  make  a  long- terra 
contribution  to  education. 

2.  The  goals  of  this  organization  are  articulated. 

3!    Our  goals  are  realistic  and  attainable  with  our  best 

efforts.  , 

4.  My  oraanization's  policy  statements  are  clear. 

5.  My  organization's  performance  standards  are  understood. 
Good  ways  are  used  to  let  me  know  how  I  can  improve 

my  performance. 

7.  I  have  good  ways  for  knowing  how  good  our  results  are. 

8.  My  manager  makes  it  clear  that  he  is  committed  to  the 
success  of  our  projects. 

9.  My  manager  has  expressed  the  belief  that  the  AMA  s 
training  program  has  been  helpful. 

10.    As  I  see  it,  planning  is  an  integral  part  of  running 
the  state's  schools. 

The  following  items  w»re  added  in  the  Y?  mestionnaire: 

13.    The  top  priority  objectives  of  state  education  are 
clear  to  me.  ^  . 

12.  I  feel  that  the  objectives  developed  during  AMA  train- 
ing reflect  the  most  serious  and  pressing  needs  of 
state  education. 

13.  As  I  see  it,  the  operational  priorities  of  the  objec- 
Y^.  tives  developed  during  AMA  trailing  are  clear. 
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14.  I  understand  what  results  must  be  produced  to  achieve 
the  stated  objectives  of  this  organization. 

15.  The  planning  unit  has  been  helpful  to  me. 

16.  fly  organization  has  reliable  ways  for  knowing  how  well 
it  is  attaining  its  objectives. 

17.  I  think  that  the  objectives  developed  during  AMA 
training  are  clearly  stated  with  respect  to  results 
expected. 

18.  My  manager  understands  planning  theory  and  is  able  to 
put  it  into  practice. 

19.  I  believe  my  organization  gives  me  adequate  training 
to  do  my  work. 

20.  I  feel  good  about  my  manager's  ability  to  plan. 

21.  My  manager  provides  me  with  adequate  support  to  perform 
my  job. 

22.  As  I  see  it,  persons  in  this  organization  put  a  lot 
of  effort  into  planning. 

23.  My  capability  to  plan  effectively  will  positively 
affect  my  future  career  in  this  organization. 

24.  The  activities  relating  to  planning  are  having  an 
effect  on  the  policy  of  this  organization. 

25.  As  I  see  it,  my  organization  is  moving  in  the  right 
direction. 

26.  My  organization's  plan  is  operable. 


The  Development  of  the  Organizational  Management  Environment 

The  following  items  have  been  continued  from  the  Yl  ques- 
tionnaire: 

1.  Based  on  information  I  have  received  from  my  boss,  I 
know  if  I  am  measuring  up  in  my  job. 

2.  My  manager  encourages  and  supports  innovation. 

3.  Higher  management's  reactions  to  the  problems  that 
reach  them  are  fair. 

4.  My  manager  knows  and  understands  the  problems  I  face. 

5.  My  manager  recognizes  when  a  problem  is  developing  and 
does  something  constructive  about  it. 

6.  My  manager  shows  confidence  and  trust  in  me. 

7.  The  people  I  work  with  participate  appropriately  in 
setting  the  goals  of  our  work. 

8.  I  am  appropriately  involved  in  decisions  affecting 
my  work. 

9.  My  group  works  hard  to  achieve  its  goals. 

10.  My  work  group  understands  what  we  are  trying  to  achieve* 

11.  I  feel  my  group  works  well  together. 

12 •    I  really  feel  my  immediate  work  group  is  getting  things 
done. 

13.    When  differences  arise  in  my  work  group,  we  have  good 
ways  for  settling  them  ourselves. 
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14.    I  can  influence  the  goals,  methods,  and  activities  of 
my  organization. 


Progress  Toward  Goal  achievement 

The  following  item  was  added  in  the  Y2  questionnaire: 

1.    As  I  see  it,  my  organization  has  made  progress  in 
attaining  its  objectives. 

A  seven  point  scale  was  the  basis  for  responses  to 
the  items  in  the  questionnaire. 


_1  2 

not  at  all 


3  4  5 

fairly  often 


very  often 


.  As  in  the  first  year's,  instrument,  an  effort  was 
made  to  expand  the  time  frame  of  the  research  data.  Two 
sets  of  responses  were  called  for  on  each  item  -  a 
perspective  on  the  organization  when  the  questionnaire  was 
being  administered  and  an  additional  perspective.    In  the 
case  of  the  Fall  1971  administration  of  the  questionnaire 
the  added  perspective  was  Fall  1970,  a  time  just  prior  to 
the  original  training.    In  the  case  of  the  Spring  1972 
administration,  the  additional  perspective  requested  was 
Fall  1972. 

This  procedure  produced  a  time-series  which  had  the 
potential  of  describing  the  change  in  respondents'  atti- 
tudes over  two  years. 

The  data  base  of  the  Y2  questionnaire  thus  is  as 
follows: 

QUESTIONNAIRE  DATA  BASE 


Agencies 
Trained 

Control 
Agencies 


Pall  '970      Fall  1971      Spring  19/2      Fall  1972 
0X0  0  0 


Projected  Back 


0  0 
Actual  Test  Admin. 


Projected  Forward 


For  '  his  report,  the  most  useful  daca  concerns  the 
points  in  time  s^hen  the  test  was  actually  administered. 
Therefore  we  have  based  our  analysis  of  Y2  on  this  data. 
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Section  6:    Administration  of  the  Questionnaire 

The  scaled  response  questionnaire  was  the  common 
instrument  used  in  all  ten  agencies  v^e  surveyed,  and  it  was 
administered  in  two  ways. 

The  procedure  followed  in  Yl  was  followed  exactly 
in  the  two  experimental  states.    The  questionnaire  was  given 
to  the  top  24  people  in  tho  state  agency  after  each  was 
interviewed.    At  that  time  the  purpose  of  the  questionnaires 
was  explained  and  the  respondent  was  asked  to  give  two 
identical  questionnaires  to  the  two  subordinates  to  whom 
he  had  given  the  questionnaire  during  the  first  year.  These 
subordinates  were  originally  identified  as  individuals  v/ith 
whom  the  respondent  had  good  communication  and  who  were  not 
involved  in  the  training.    This  procedure  produced  a  test 
population  o£  72  in  each  state.    As  in  the  first  year's 
evaluation,  the  assumption  was  made  that  "good  communication" 
meant  that  the  individual  thus  identified  would  be  to  some 
degree  sensitive  to  the  influence  of  the  training.  This 
assumption  seems  even  more  valid  for  the  present  study,  for 
the  test  organizations  have  since  developed  and  conducted 
in-service  training  programs  based  on  what  they  learned 
from  the  AMA. 

In  the  control  states,  the  questionnaire  was  mailed 
to  a  pre-arranged  contact  in  each  agency  who  then  distributed 
the  questionnaire.     In  control  state  1  the  population  was 
identified  as  being  the  same  as  the  one  tested  in  year  one. 
In  the  control  state  added  for  the  second  year's  evaluation 
the  state  superintendent  v>as  asked  to  identify  the  24 
persons  who  were  felt  to  be  the  most  important  in  terms  of 
the  operation  of  the  agency.    This  was  the  same  procedure 
used  by  the  AMA  in  selecting  persons  to  be  trained  and  in 
testing  control  state  1.    Again,  the  total  potential 
population  which  completed  the  questionnaire  in  each  control 
St ace  was  seventy- two. 

The  procedure  we  followed  in  the  local  educational 
agencies  was  identical.    However,  because  only  12  top 
administrators  were  trained  in  each  of  the  pilot  LEA's 
the  total  population  in  each  LEA  was  thirty-six — twelve 
administrators  and  twenty-four  subordinates. 

The  questionnaires  were  administered  circumspectly 
so  as  to  insure  anonymity.     Instructions  on  the  cover  page 
of  the  questionnaire  asked  respondents  to  return  the  ques- 
tionnaire to  the  contact  person  in  a  sealed  envelope,  and 
a  plain  brown  envelope  was  distributed  with  each  question- 
naire for  this  purpose. 
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Section  7;    Statistical  Analysis  of  the  Questionnaire  Data 

The  major  problem  in  statistical  analysis  of  the 
questionnaire  data  was  to  select  a  statistic  that  was  both 
comprehensive  and  powerful.    Another  important  problem 
was  to  determine  simultaneously  to  what  extent  the  compared 
groups  were  similar  or  dissimilar  and  to  what  extent  the 
training  was  producing  change  as  measured  by  the  question- 
naire items. 

The  statistical  test  we  chose  to  analyze  this  data 
was  the  Two-Way  Analysis  of  Variance,  the  same  statistic 
used  in  the  Yl  evaluation.    Conventionally,  this  statistic 
can  be  designed  in  two  ways.    One  design  enables  a  compari-- 
son  of  row  and  colvunn  variance  plus  a  test  for  interaction 
between  rows  and  columns. ^    V7e  chose  the  second  design 
here  on  the  grounds  that  it  was  important  to  know  whether 
the  interaction  of  training  effects  and  differences  between 
States  were  significantly  influencing  cinalysis  of  the  row 
and  column  data.    In  essence,  this  design  checks  the  selec- 
tion-maturation problem  which  was  questicneibly  controlled 
by  the  original  research  design  (Cf.  Chapter  Two) • 

This  particular  statistic  (1)  tests  for  the  existence 
of  significant  differences  between  the  States  and  LEA's 
which  are  being  compared  (this  test  is  made  on  the  basis 
of  both  the  mean  scores  and  the  variances  around  each  of  these 
means);   (2)  tosts,  on  the  same  grounds — means  and  variance 
around  the  meems—for  the  effects  of  training  on  the  basis 
of  before  and  after  scores. 

The  first  of  these  tests,  in  effect,  holds  time 
constant,  and  answers  the  question,  "Tire  the  groups  statis- 
tically different  from  each  other?"    Th^s  is  represented 
by  row  variance  and  is  summarized  by  the  value  of  the  F 
statistic.    The  other  analysis,  column  variance,  examines 
for  differences  over  time  and  provides  an  emswer  to  the 
question,  "Did  the  training  program  have  a  statistically 
significant  effect?"    Throughout,  the  .05  level  of 
significance  is  the  minimum  basis  for  the  decision  that 
training  did  indeed  produce  a  dif f erence. 1^ 


Herbert  Blalock,  Social  Statistics  (New  York: 
McGraw-Hill,  1960),  pp.  253-266»    Cf.  especially  p.  264. 

^^In  case  the  reader  is  not  familiar  with  this 
terminology,  the  .05  levf^l  of  significance  means  that  if  we 
say  a  difference  exists  between  groups,  we  would  expect  to 
bo  incorrect  in  making  this  inference  only  five  times  out 
of  a  hundred.    By  the  phrase  greater  significance  v/e  mean 
the  *01  or  .001  level  which  means  we  would  expect  to  be 
incorrect  in  saying  a  difference  existed  between  the  groups 
only  once  out  of  one  hundred  times  or  once  out  of  one 
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Section  8t    Summary;    Statistics  Used  to  Analyze 
QuestionnairG  Material 

In  the  context  of  this  study,  analysis  of  column 
variance  is  thus  an  analysis  of  the  effects  of  training. 
The  analysis  of  column  veiriance  will  tell  us  if  change 
occurred  between  the  scores  obtained  in  two  dlTferent 
periods. 

Row  variance  moans  sr>  analysis  of  differences  ' 
between  the  states,  independent  of  changes  that  occurred 
over  time  within  the  states.    This  statistic  tells  us  if 
the  states  differ  from  each  other  in  the  degree  of  emphasxs 
given  a  particular  item  when  change  in  emphasis  given  to 
the  itom  is  held  constant  and  our  only  concern  is  the  amount 
of  emphasis  in  each  state  for  a  specified  time  period. 

With  two  exceptions,  we  evoke  only  one  research 
design  in  the  analysis  of  the  questionnaire  data.    This  is 
the  Bon- equivalent  Control  Group  Design  discussed  in 
Chapter  Two  of  this  report.    This  design  ap:^ies  to  all 
the  questionnaire  items  continued  from  the  VI  questionnaire, 
for  which  we  have  data  at  four  times,  including  a  pre-train- 
ing  test.    The  meaning  of  thi?  data  is  straightforward 
and  Ccin  be  confidently  interpreted. 

The  first  exception  involves  four  items  which  were 
not  asked  in  the  control  states  because  they  queried 
attitudes  related  to  AMA  training  and  were  applicable  only 
to  managers  in  the  experimental  states.    In  these  instances 
we  are  faced  with  a  weak  research  design  because  of  the 
absence  of  a  control  group.       ^^en  this  problem  is  encoun- 
tered in  the  text  we  remind  the  reader  of  it. 

The  second  exception  involves  items  which  were  added 
to  the  Y2  questionnaire  and  thus  allow  only  a  T3-T4  compari- 
son.   No  pre-training  data  exists  to  be  used  with  this  data. 
We  must  therefore  be  much  more  cautious  in  interpreting 
this  material. 12    However,  because  a  second  control  group 
was  added  in  T3  and  because  wo  can  draw  rather  direct 
inferences  from  the  T1-T4  comparisons  and  the  organizational 
documents  to  support  the  results  of  this  data,  v;g  have  made 
judgments  concerning  training  effects  based  on  this  data. 
Again,  the  reader  will  be  reminded  when  such  qualified  data 
appears  in  the  text. 


tnousand  times  respectively.    The  greater  the  significance 
the  greater  the  probability  of  a  correct  decision. 

^^Cf.  Chapter  Two  of  this  report  for  a  complete 
discussion  of  the  problems  related  to  this  type  of  research 
design. 

O  ^^Cf.  Chapter  Two  -  Research  Methodology. 

ERIC 
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Section  9;    The  Use  of  Interviews 

Unlike  questionnaires,  interviews  require  a  tremen- 
dous investment  of  time  by  researchers  and  respondents. 
There  ere  also  difficult  and  time  consuming  tasks  which 
must  be  completed  to  utilize  the  data  thus  collected. 
Those  include  developing  across-the-board  content  cate- 
gories to  analyze  data  which  does  not  fall  neatly  into 
categories,  since  interviewees  are  asked  to  respond 
spontaneously  to  whatever  questions  are  posed.    For  pur- 
poses of  coding,  a  structural  or  forced-choica  interview 
mode  is  usually  suggested. 

Disadvantages  are  outweighed  in  this  evaluation 
situation,  however,  by  advantages.    Backstrom  and  Harsh 
point  out  that  the  free-response  question  is  especially 
useful: 

(1)  where  the  researcher  has  limited 
knowledge  as  to  the  kind  of  answers  a 
particular  question  is  likely  to  pro- 
voke, (2)  where  he  anticipates  a  gre^t 
range  of  responses,   (3)  where  he  is 
interested  in  what  the  respondent  will 
volunteer  on  a  subject  before  specific 
prompting,  or  (4)  where  he  wants  to  go 
a  little  deeper  into  respondents' 
motivations. 13 

This  free  response  interview  technique  has  compara- 
tive advantages  over  the  imposition  of  controls.  The 
interviewer  can  provide  the  overall  framework  for  the 
interview  by  asking  a  set  number  of  basic  questions.  But 
any  follow-up  questions  are  based  on  the  responses  of  the 
interviewee.    As  such,  the  interview  techniques  ado^^ted 
for  Yl  and  Y2  allow  us  to  be  more  confident  of  the  findings 
and  conclusions  generated  by  the  intexrview  dnta. 

As  the  first  year's  report  made  clear,  the  semi- 
structured  interview 

enables  the  respondent  to  describe 
circumstances  cind  events  with  a 
minimal  amount  of  definitional  struc  ' 
ture  provided  b]^  the  researcher. 
Theoretically,  material  produced 
through  this  method  will  be  more 
••reality  oriented";  more  as  the  inter- 
viewee sees  and  defines  things.  There 


Backstrom  and  Hursh,  0£.  ext.,  p.  73. 
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is  also  reason  to  believe  that  data 
gathered  through  this  technique  will 
be  more  conservative,  i.e.,  less 
likely  to  show  training  effects  and 
that  when  effects  are  produced  they 
are  more  likely  to  be  of  meaning  and 
value  to  the  respondent  and,  hence, 
the  organization  .  .  .  the  semi- 
structured  interview  process  is  more 
likely  to  reveal  internalized  beliefs 
held  by  the  respondent.  •'■^ 


Section  10;    Administration  of  the  Interviews 

The  procedure  followed  in  conducting  the  interviews 
duplicated  that  of  the  first  year's  research.    The  top 
twenty-four  administrators  chosen  by  the  State  Superintendent 
to  participate  in  the  training  were  interviewed  in  the  two 
experimental  states.    The  top  twelve  administrators  were 
interviewed  in  Control  State  1.    As  in  the  first  year,  the 
analysis  of  the  interviews  is  restricted  to  the  top  twelve 
administrators  in  each  state  educational  agency. 

With  one  exception,  interviews  were  conducted 
exactly  as  they  had  been  during  the  Yl  research.  Inter- 
views took  place  on-site  in  the  organizations  themselves, 
in  settings  which  insured  maximum  privacy.    A  slightly 
different  format  was  followed  with  the  Control  State  in  ¥2. 
Because  of  sch<^,duling  difficulties  and  the  fact  that  the 
Control  State  was  entering  training  in  the  Pall  of  1971, 
the  research  team  conducted  the  first  set  of  interviews 
at  the  AHA  training  site  in  Hamilton,  New  York,  at  night 
and  in  the  privacy  of  respondents*  rooms.    The  second  set 
of  interviews  were  conducted  on-site  in  each  organization. 

Each  respondent  in  all  interviews  was  asked  twelve 
questions  in  the  Pall  and  fifteen  questions  in  th ,  Spring. 
They  were  assured  of  their  anonymity  and  urged  to  be  as 
open  and  cemdid  as  possible. 

The  questions  continued  from  the  Yl  evaluation  were: 

1.  What  do  you  think  you  got  out  of  the  training  experi-- 
ence  with  AMA? 

2.  How  are  major  decisions  made  in  the  State  Department? 

3.  What  is  tha  role  of  planning  in  running  the  state's 
schools? 

4.  How  do  you  feel  about  the  direction  your  organization 
is  moving? 


Kirkhart  and  Tanner/  o£.  cit . ,  pp.  37-38. 
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5.  What  are  the  roadblocks  to  change  in  this  organiza- 
tion? 

The  questions  asked  in  the  Fall  and  Spring  of  Y2  were: 

6.  Do  you  feel  that  the  objectives  developed  during  AMA 
training  reflect  the  most  serious  and  pressing  needs 
of  state  education? 

7.  Were  there  any  people  or  groups  whom  you  feel  should 
have  participated  in  the  development  of  objectives 
for  the  state  department  of  education  who  did  not 
participate? 

8.  Is  planning  influencing  the  decisionmaking  process 
within  the  state  department  of  education? 

9.  How  has  the  planning  unit  helped  you  to  plan? 

10.  Toward  what  action  plem  objectives  has  measurable 
progress  been  made  by  your  division? 

11.  Toward  what  action  plan  objectives  has  measurcible 
progress  been  made  by  the  state  department  as  a  whole? 

12.  What  changes  in  the  planning  process  or  in  the  action 
plan  itself  do  you  feel  are  necessary  at  this  time? 

The  questions  asked  in  the  Spring  of  Y2  were: 

13.  What  specific  methods  do  you  use  to  determine  if  the 
continuing  and  specific  objectives  of  your  division 
are  being  met? 

14.  Have  performance  standards  been  established  for  your 
subordinates  based  on  the  objectives  in  your  division's 
plan? 

15.  Do  you  have  regular  performance  reviews  with  your 
subordinates? 

The  interviewers  worked  1-jm  a  set  of  cards  which 
conta'ied  the  questions  and  insured  that  the  order  of 
items  vas  always  the  same*    After  the  initial  response, 
non-directive  probes  were  used  to  draw  out  the  respondent 
and  insure  that  he  had  responded  to  the  question  as  fully 
as  he  could  or  wished  to  do.    Feedback  techniques  were 
used,  ive.,  "I  hear  you  saying  that..."  or  "the  major 
points  you  are  making  are...."    This  technique  served  to 
elicit  additional  infoxrmation  and  also  to  correct  and 
clarify  impressions  the  respondent  was  making. 


Section  11:    Analysis  of  the  Semi-Structured  Interview  Data 

Tape-recorded  material  from  the  interviews  produced 
approximately  twelve  pages  of  double-spaced  typescript  per 


Before  the  interview  process  was  begun  both  inter- 
viewers went  through  a  training  session  to  improve  and 
practice  their  ability  to  provide  non-directive  feedback. 
This  training  was  provided  by  the  faculty  advisor  to  this 
pro  j  ect.   ^   ^ 
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respondent.    ThG  material  was  subjected  to  content  analy- 
sis and  coded, 16  a  procedure  that  involved  organizing  the 
material  in  such  a  way  that  ans\#ers  were  provided  to 
specific  questions  being  researched.    A  random  sainple  of 
the  interviews  were  reviewed  and  a  set  of  categories  was 
developed  for  each  interview  item.^'    A  seven  point  scale 
was  used  to  quantify  the  responses. 

We  have  used  twelve  of  the  interview  questions  in 
the  data  base  of  this  study.    The  questions  and  the  content 
categories  we  use  in  this  report  are  listed  below. 


Interview  Questions  and  Content  Analysis  Categories 

A.  What  do  you  think  you  got  out  of  the  training  experience 
with  AMA? 

1.  Definition  of  the  institution's  mission 

2.  Modify  previously  established  objectives 

3.  Determine  priorities 

4.  Identify  and  analyze  alternative  courses  of 
action 

5.  Define  standards  of  performance  for  key  admin- 
is  trators 

6.  Specify  tark  completion  dates  and  action 
assignments 

7.  Assign  responsibilities  to  subordinate  units 

8.  Design  a  methodology  by  which  future  performance 
may  be  evaluated  in  relation  to  the  performances 
specified  in  the  plan 

9.  Produce  and  implement  a  long-range  strategic 
plan 

10.  Establish  credibility  of  planning 

11.  Promote  cooperative  team  work 

B.  Do  you  feel  that  the  objectives  developed  as  a  result 
of  AMA  training  reflect  the  most  serious  and  pressing 
needs  of  state  education? 

11.     (Question  acts  as  a  content  domain  -  no  sub- 
classification  ;:ecessary) 


^^CC.  Robert  North/  Content  Analysis  (Evanston:  North- 
western Univ.  Press,  1963);  Ole  R.  Holsti,  Content  Analysis 
for  the  Social  Sciences  and  Humanities  (Reading^  Mass.: 
Addison-Wesley,  i$69) ;  and  Bernard  Berelson,  Content  Analy- 
sis  in  Communication  Research  (Glencoe,  111.:    Free  Press, 

rmr.  


The  complete  coding  document  is  found  in  Appendix 
of  this  report. 
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C.  How  are  major  decisions  made  in  the  State  Department? 

12.  Involvement  in  decisionmaking 

13.  Quality  (effectiveness  of  decisionmaking) 

D.  What  is  the  role  of  planning  in  running  the  state's 
schools? 

14.  Role  cf  planning  (how  integral  is  it?) 

15.  Need  for  planning  (how  much  is  needed?) 

16.  Emergence  of  planning  (when  it  became  an  issue?) 

E.  Is  planning  influencing  the  decisionmaking  process 
within  the  State  Department  of  Education? 

17.  (Question  acts  as  a  content  domain  -  no  sub- 
classification  necessary) 

F.  How  h;^s  the  planning  unit  helped  you  to  plan? 

18.  Awareness  of  need  to  evaluate  our  plans 
19 •    Available  to  answer  planning  questions 

20.  Reviewing  and  refining  plans 

21.  Provides  leadership  in  the  implementation  of 
planning 

22.  Provides  in-service  training  in  planning 

G.  Toward  what  action  plan  objectives  has  measurable 
progress  been  made  by  your  division? 

23.  Number  of  objectives  toward  which  progress  has 
been  made 

24.  Level  of  progress  toward  those  objectives 

H.  How  do  you  feel  about  the  direction  your  organization 
is  moving? 

25.  (Question  acts  as  a  content  domain  -  no  sub- 
classification  necessary) 
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I.    What  are  the  roadblocks  to  change  in  this  organization? 
26*    Adequate  Resources 

27.  Control  System  expressed  through  decisionm2king 
process 

28.  S^^nse  of  SED  mission 

29.  Aiiount  of  cooperative  teamwork  present 

^  J.    Have  performance  standards  been  established  for  your 


subordinates  based  on  the  objectives  in  your  division 
plan? 
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30 •    Extent  of  Use 

31.  Need  for  Performance  Standar  s 

K.    Do  you  have  regular  performance  reviews  with  your 
subordinates? 

32.  Existence  of  Performance  Reviews 

L.    What  specific  methods  do  you  use  to  determine  if  the 
continuing  and  specific  objectives  of  your  division  are 
being  met? 

33.  Performance  Reviews 

34.  Questionnaires 

35.  Task  Completion  Inventories 

36.  Unobtrusive  Measures 


Section  12;    Procedures  Used  in  Content  Coding 

The  validity  of  content  coding  depends  heavily  on  a 
common  understanding  among  the  coders.    To  deal  with  this 
problem  in  the  Yl  evaluation,  the  field  researchar  for  Yl 
and  the  two  members  of  the  Y2  research  team  worked  jointly 
to  develop  the  content  categories  and  to  code  the  inter- 
views.   Given  this  previous  experience ,  the  Y2  research 
team  coded  the  interviews  they  completed  in  the  second 
year  of  research. 

In  order  to  insure  that  this  material  would  be 
treated  as  objectively  as  possible,  we  made  every  effort 
to  develop  a  mutual  understanding  of  the  material  and  the 
way  it  was  to  be  coded.    Several  trial  runs  were  undertaken 
in  which  each  coder  independently  coded  the  same  interview 
and  then  compared  his  product  to  assure  a  high  degree  of 
similarity  in  each  coder's  procedure.    Once  similarity  was 
attained/  the  entire  body  of  interviews  was  coded • 

The  coders  read  the  entire  interview  document  prior 
to  coding.    This  was  done  to  avoid  the  Assumption  that  the 
interviewee's  verbal  response  always  proceeded  in  an  exactly 
logical,  sequf?'*tiai  manner*    Thus  we  could  incorporate 
remarks  that  were  appropriate  to  an  earlier  section  of  the 
interview  but  were  articulated  only  later-    We  intended 
to  give  the  respondent  every  opportunity  to  provide 
recordable  material  for  the  research;  since  this  meant,  in 
many  instances,  maximum  comment  on  matters  directly  related 
to  the  goals  of  the  AMA  project,  this  also  means  that,  if 
anything,  a  positive  bias  exists  in  the  scores  we  recorded. 
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Section  13:    Statistical  Analysis  of  the  Content  Material 

Because  the  number  of  subjects  in  this  analysis 
is  small,  it  was  necessary  to  select  statistical  tests 
expressly  designed  for  small  samples-^-non-parametric 
statistics*    Since  the  semi** structured  interview  is 
designed  to  enable  the  respondent  to  project  his  or  her 
own  definition  of  the  situation  onto  the  research  question, 
the  number  of  respondents  to  each  of  the  research  categories 
defined  by  the  coding  instrument  fluctuates  considerably. 
This  fluctuation  provides  one  point  of  analysis;  the 
scaling  technique  which  recorded  intensity  of  response 
provided  a  second  point  of  analysis.    In  order  to  test  for 
differences  in  intensity  of  reaction,  the  Rruskal^-'Wallis 
One*Way  analysis  of  variance  was  applied  to  the  scale 
scores.    This  test,  capable  of  handling  extremely  small 
numbers  of  respondents,  still  provides  a  meaningful  analysis 
of  the  probability  of  differences  between  groups. 

The  procedure  utilized  in  this  test  is  to  pool  the 
scores  of  individuals  from  both  groups  and  then  rank  this 
total  set  of  data  from  high  to  low.    Each . individual  score 
is  translated  into  an  ordered  ranking  in  which  the  highest 
individual  score  in  the  pool  receives  the  lowest  numerical 
score.    In  other  words,  the  individual  whose  response  was 
highest  would  receive  the  ranking  of  one,  the  individual 
with  the  second  highest  response  would  receive  a  two,  etc., 
until  all  individuals  have  been  ranked.    Then  this  aggre* 
gate  data  which  is  composed  of  responses  from  both  groups 
is  redistributed  back  to  individual  group  rankings.  Using 
a  comparison  of  the  strength  of  these  rankings  in  each 
group,  a  decision  can  be  made,  based  on  probability,  about 
whether  the  groups  are  or  are  not  differe^^t. 

The  Kruskal-VJallis  test  thus  enables  us  to  decide 
which  group  placed  the  greatest  emphasis  on  the  research 
category.    This  test  is  not  aTTected  fay  the  number  of 
respondents  in  each  group;  it  simply  reveals  (given  any 
number  of  respondents)  whether  the  degree  of  emphasis 
differs  between  the  groups .    The  test  statistic  which 
provides  this  information  is  the  H  statistic.    Only  when 
this  value  reaches  the  .OS  level  of  significance  will  we 
say  that  a  difference  existed  between  the  subject  groups. 

A  second  test  was  included  to  examine  the  Important 
question  of  what  kinds  of  fluctuations  occurred  in  the 
number  of  respondents  and  whether  or  not  these  fluctuations 
were  significant  between  groups.    The  semi-structured 


Cf.,  Sidney  Siegel,  Non*Parametric  Statistics  for 
the  Bohayioral  Sciences  (Mew  York:    McGraw-Hill,  1956), 
pp.  184-93. 
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interview,  designed  to  enable  the  respondent  to  say  what 
was.  important  to  him  at  that  particular  time,  makes  this 
question  very  significant.    If  the  training  program  had 
any  impact  on  attitudes/  a  larger  number  of  people  would 
possibly  be  aware  of  specific  issues  after  the  training 
than  were  aware  of  the  same  issues  before  training. 

The  question  we  wanted  to  explore  was,  "Did  the 
training  have  the  effect  of  changing  the  population  of 
people  who  were  aware  of  specific  issues?"    Since  we  cou^d 
answer  the  question  of  changing  emphasis  by  the  test  pre'- 
viously  described/  we  wanted  to  determine  if,  independent 
of  intensity  of  reaction/  aggregate  awareness  changed. 
The  Binomial  Test  of  Proportions  provides  this  informa- 
tion.^^   It  provides  a  useful  analysis  so  long  as  the 
population  is  less  than  twelve;,  in  those  cases  where  all 
of  the  persons  interviewed  provided  information  relevant 
to  the  research  category/  the  test  has  no  meaning;  aware- 
ness of  the  issue  already  existed  for  all  people. 

The  symbol  used  in  the  text  to  represent  the  Binomial 
Test  will  be  a  P;  only  when  its  value  is  such  that  the  .05 
level  (or  a  greater  difference)  is  obtained  will  we  say 
that  a  difference  existed  between  the  groups  studied. 


Section  14;    Summary:    Statistics  Used  to  Analyze  the 
Interview  Content  Data 

The  text  relies  heavily  on  the  Kruskal-Wallis 
statistic;  in  a  number  of  instances  the  test  of  proportions 
also  will  be  used.    The  first  of  these  tells  us  whether  a 
significant  change  appears  in  the  degree  of  emphasis  given 
to  a  particular  research  category  when  one  group  is 
compared  to  another.    The  second  test  tells  us  whether  a 
significant  increase  exists  in  the  degree  of  aggregate 
awareness,  independent  of  intensity  expressed,  which  can 
be  attributed  to  the  effect  of  training  or  which  existed 
between  the  groups. 

Finally/  it  should  be  pointed  out  that  interpreta- 
tion of  these  statistics  is  somewhat  ambiguous  in  some 
instances.    Because  interview  questions  which  were  directly 
related  to  expected  or  actual  experience  with  the  AMA 
training  program  were  only  relevant  to  the  two  Experimental 
States,  we  faced  the  problem  of  a  weak  research  design 
whenever  these  questions  were  encountered  elsewhere.  As 
was  discussed  above,  in  such  instances  we  grappled  with  a 
research  design  which  lacked  a  Control  group. 


Blalock,  0£.  cit. ,  pp.  176-77. 
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Whenever  data  of  this  type  is  encountered,  its 
interpretation  should  be  regarded  as  more  tenuous  than  is 
the  case  when  a  Control  group  comparison  is  included. 
The  reader  will  be  reminded  of  this,  too,  when  such  data 
appears. 


Section  15:    The  Use  of  Organizational  Docximents 

A  great  variety  of  organizational  documents  have 
been  searched  and  reviewed  to  offer  background  to  the 
evaluation  and  the  evaluators.    Most  evaluators  agree  that 
"mere  collection  of  data  does  not  constitute  evaluation. "20 
This  information  has  served  as  qualitative  support  for 
our  conclusions  when  they  did  not  lend  themselves  to  total 
quantification  or  when  their  accuracy  could  not  be  guaran- 
teed by  the  research  design  itself.    As  we  have  indicated 
earlier,  because  of  nonrandom  selection  of  the  experimental 
population  and  the  lack  of  conclusive  proof  of  pre-sampling 
equivalence,  judgment  must  lead  the  way  into  areas  unmapped 
by  statistical  analysis. 

Organizational  documents  offer  another  back-up 
indicator  to  the  interviews  and  questionnaires  which  will 
lend  greater  credence  to  our  findings.    Frequently,  a 
particular  evaluation  of  the  effect  of  a  program  based  on 
one  or  two  indicators  will  be  contradicted  or  amended  if 
another  indicator  is  taken  into  account;  different  indi- 
cators give  different  perspectives  on  the  same  program. 

We  suggest  that,  as  a  general  rule,  any 
measurement  of  a  social  science  concept 
that  relies  on  a  single  indicator  should 
be  viewed  as  dubious.    While  simply  add- 
ing more  indicators  is  of  little  value 
if  they  measure  the  same  dimension,  draw- 
ing on  two  or  more  indicators  of  differ- 
ent dimensioriS  provides  at  least  partial 
insurance  against  fractional  coverage 
and  its  dysfunctions. 

Searches  of  orgcinizational  documents  can  be  more 
quickly  done  than  administration  and  analysis  of  tests. 
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^^Egon  G.  Cuba,  "The  Failure  of  Educational 
Evaluation,"  Educational  Technology  (May,  1969). 

^^Amitai  Etzioni  and  Edward  w.  Leham,  "Some  Dangers 
in  'Valid*  Social  Measurement,"  The  Annals,  373  (September, 
1967),  p.  4. 
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One  of  the  many  drawbacks  to  quantitative  analysis  is  that 
its  results  come  too  long  after  the  conclusion  of  the 
evaluation  to  be  useful  in  making  methodological  changes. 
With  y2,  we  have  been  able  to  use  organizational  documents 
to  focus  other  instruments  on  crucial  areas.    The  time 
lag  between  data  collection  and  judgment  about  the  effec- 
tiveness of  instruments  to  measure  what  should  be  measured 
has  been  abbreviated  slightly* 

Finally^  as  Campbell  has  written^  organizational 
documents  enable  prior  records  of  the  experimental  group 
to  serve  as  the  control  or  the  basis  for  inferring  what 
would  have  happened  without  the  intervention  of  the  train- 
ing program. 22    por  example^  if  an  LEA  had  not  published 
a  comprehensive  plan  or  cost  benefit  analysis  for  several 
years  before  the  AMA  program,  and  then  in  the  year 
subsequent  produced  even  a  few  such  documents,  such  a 
change  may  reasonably  be  connected  to  the  training  program. 
Or  if  an  SED  established  a  planning  unit  immediately  after 
the  training  program  where  none  had  existed  previously, 
the  AMA  could  claim  at  least  partial  credit. 


Section  16:    Analysis  of  Organizational  Documents 

Y2  has  relied  on  organizational  documents  as 
indirect/  and  occasionally  diract,  evidence  of  th3  impact 
of  AMA  training.    Such  papers  include:     (1)  action  plans 
developed  by  SED's  and  LEA's  during  AMA  training,   (2)  plans 
produced  by  SED's/LEA's  subsequent  to  draining  either  to 
amend  existing  action  plans  or  to  elaborate  on  them,  (3) 
attempts  to  implement  AIIA- suggested  preconditions  for 
effective  planning  such  as  in-service  training,  agency- 
wide  job  descriptions,  etc.,   (4)  revisions  in  the  organi- 
zational chart  (such  as  the  establishment  of  a  planning 
unit)  as  indirect  evidence  of  pre/post  training's  formal 
emphasis  on  planning,   (5)  previous  consulting  reports  on 
the  strengths  and  v/eaknesses  of  the  experimental  SED/LEA*s, 
and  (6)  miscellaneous  evidence  of  planning  output,  e.g., 
use  of  indices  of  goal  attainment,  input  of  client  groups 
as  shown  by  samples  and  feedback  from  LEA  officials,  etc. 


Donald  T.  Campbell,  "Considering  the  Case  Against 
Experimental  Evaluations  of  Social  Innovation,"  Administra 
tive  Science  Quarterly,  15:1,  pp*  110-113.    See  also: 
"Reforms  as  Experiments." American  Psychologist,  24,  pp. 
409-429  by  the  sane  author. 
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PART  III 
FINDINGS  AND  ANALYSIS 
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INTRODUCTION 


Our  central  concern  in  this  evaluation  is  to 
measure  the  impact  of  the  AMA's  training  urogram  over  time 
^!Z-i"^       context  of  the  organizational  environment  of 
attitudes  and  actions  inside  the  experimental  organizations. 
The  essential  questions  wo  attempt  to  answer  here  are  "To 
what  extent— if  any— did  the  a:ia  program  change  the  atti- 
tudes and  actions  of  the  people  in  the  experimental   

organizations?    mat  effect  on  organizational  output  did 
any  such  change  exert?"    To  be  considered  effective  the 
training  program  must  have  eventually  improved  the  output 
of  the  organizations  trained. 

Our  analysis  of  attitudinal  change  is  based  on  the 
assumption  that  people  in  organizations  act  on  the  basis 
of  a  complex  network  of  beliefs,  values,  norms,  and  defini- 
tions of  reality  that  are  peculiar  to  the  organization. 
Based  on  the  stated  premises  of  the  AMA  (discussed  below) 
we  assume    that  the  training  program  attempted  to  change 
sane  of  these  attitudes. 

We  do  not  assume  that  a  direct  relationship 
necessarily  obtains  between  expressed  beliefs  and" actual 
behavior  or  that  beliefs  and  values  held  by  persons  in  an 
organization  are  always  consistent  with  their  overt 
behavior. 


Attitudinal  change  is  a  necessary 
condition  to  changing  organizational 
behavior  but  not  a  necessary  and  suf- 
ficient condition.    Social  realTtyTn 
an  organization  is  considerably  more 
complex  and  is  in  all  cases  conditioned 
by  perceived  possibilities  of  action 
"in-the-situation."    These  possibilities 
are  .  .  .  shaped  by  the  environment 
(political,  economic,  social)  of  the 
organization,  relationships  between  and 
among  departments,  group  norms  or 
standards,  action  possibilities  created 
by  technology,  and  the  orientation  of 
2.ndividual.'s  to\*ard  organizational 
processes. 
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sity, October,  1971),  p.  40. 


84 


Our  emalysis  of  impact  related  to  action  is  based  on  tho 
program's  objectives  set  forth  by  the  ?Jtirv.    !'Je  are 
specifically  interested  in  the  effects  of  time  and  organi- 
zational environment  on  the  plans  developed  by  the  experi- 
mental organizations  during  AIITV  training.    To  what  extent 
have  the  continuing  and  specific  objectives  of  the  plans 
produced  during  and  subsequent  to  training  been  achieved? 
We  are  interested  too  in  the  plans  produced  and  in  measured 
progress  toward  the  achievement  of  the  stated  objectives. 

Before  discussing  the  conceptual  linkage  used  to 
evaluate  the  impact  of  the  Ar!A's  program,  it  will  be  useful 
to  review  the  basic  premises  and  training  approach  employed 
by  the  MIA  in  its  management  development  programs  for 
educational  administrators.    What  does  AI^A  perceive  to  be 
management  problems  in  educational  agencies #  and  what  do 
they  propose  to  do  about  thcm?'^ 


The  AMA  Approach:    Basic  Premises 

The  AMA  recognizes  that  executives  generally  lack 
sufficient  training  in  managerial  principles  and  practices. 
This  shortcoming  is  evinced  by  many  kinds  of  organizational 
enterorises/  not  only  businesses.    Persons  promoted  into 
high  administrative  positions  on  the  basis  of  past  performance 
often  cannot  cope  with  the  increased  pressures  of  those 
positions.    According  to  the  AMA,  the  reasons  for  executive 
failure  often  include  the  fact  that  "seldom  is  there  any 
attempt  to  ensure  that  appropriate  training  or  experience 
in  management  is  part  of  one's  qualifications.''-^ 


The  information  for  this  section  was  gathered  from 
four  basic  sources:  1)  American  Management  Association^ 
"Feasibility  and  Pilot  Programs  Proposal:    Adapting  and 
Testina  Business  Management  Development  Programs  for  Educa- 
tional" Administrators"  (June  22,  1970)  (flimeo. ) ;  2)  Kirkhart 
and  Tanner,  o£.  cit. ;  3)  Raymond  E.  Klawuhn  and  Alexander 
J.  Basso,  "AcIaptTng  and  Testing  Business  Management 
Development  Programs  for  Educational  Administrators,"  Final 
Report,  Vol.  1  (January,  1972);  and  4)  Informal  discussions 
with  personnel  in  the  American  Management  Association  and 
the  experimental  educational  agencies. 

Klawuhn  and  Basso,  0£.  cit. ,  P*  1* 
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An  ongoing  concern  of  the  A^i\  has  been  to  improve 
nanagcrial  effectiveness  in  business  and  industry.  After 
an  extensive  examination^  tho  AMA  concluded  that  many  of 
the  problems  thoy  had  long  been  handling  in  their  corporate 
clients  vteve  also  obvious  in  educational  agencies.  Because 
the  problems  were  similar  the  solutions  could  also  be 
similar.    In  the  vrords  of  the  AMA:^ 

1.  The  management  problems  facing 
educational  agencies  and  institu- 
tions are  analogous  to  those  facing 
business  and  industrial  enterprises. 

2.  The  management  skills  and  techniques 
practiced  by  business  and  industrial 
enterprises  could  be  modified  and 
effectively  applied  by  managers  of 
educational  agencies. 

3.  The  management  and  organizational 
development  programs  of  the  AMA 
could  be  modified  and  adapted  to 
the  particular  requirements  of 
educational  managers^  thus  enhancing 
the  management  of  educational  agen- 
cies and  institutions. 


The  Aim  Perspective  on  Ilanagement  Problems  in  Education 

In  their  final  AIIA  report  on  the  pilot  project  being 
evaluated  here,  five  basic  management  problems  that  affect 
the  performance  of  educational  agencies  are  summarized: ^ 

1.  Decis  ionmaking  proces  ses ;    Most  edu- 
cational agencies  lack  a  coherent , 
explicit  plan  of  action.    Without  the 
guidelines  such  a  plan  would  provide , 
the  decisionmaking  process  within  the 
agencies  is  often  nonsystematized  and 
vague.    Administrators  never  really 
know  what  the  decisionmaking  process 
of  their  respective  agency  is^,  let 
alone  v;hat  it  should  be. 

2.  Organizational  Structure:    The  compara- 
tiVG  responsibilities  of  the  various 


^Ibid. ,  p.  2. 

^AMA,  0£.  cit. ,  pp.  3-4;  Klav\mhn  and  Basso,  o£.  cit. , 
pp.  3-4. 
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levels  within  an  educational  system 
are  frequently  left  undefined.  The 
powers  and  duties  of  school  boards / 
superintendents,  principals/  and 
staff  specialists  result  frc»n  long-- 
standing tradition  emd  legal  provi- 
sions rather  than  any  conscious  attempt 
at  organization.    Job  descriptions  and 
performance  standeirds  for  key  adminis- 
trators either  do  not  exist  or  are  un- 
clear «    As  a  result,  they  are  unsure 
of  what  is  expected  of  them  and 
responsibility  becomes  diffused.  The 
agency  then  becomes  unable  to  deal 
with  its  current  problems  or  future 
chemges. 

3*    Converting  educational-theory  into 
practice:    Recent  advetnces  in  educa- 
tional theory  which  could  have  long- 
term  benefit  for  educational  agencies 
are  untranslated  into  general  practice. 

4«    Integrated  Planning:    Various  agencies 
ana  levels  within  the  same  educational 
system  work  in  isolation*  Consequently 
their  plans  beccxne  unrelated  and  system 
planning  becomes  f ractionalized.  Each 
part  "does  its  own  thing"  so  the  whole 
becomes  divided  and  lacks  coherent 
direction.    In  addition,  means  and  ends 
becc^e  confused:    strategies  for  achiev- 
ing objectives  are  accepted  as  objec- 
tives in  themselves  and  short-term, 
perhaps  temporary,  gains  are  mistaken 
for  satisfactory  end  results. 

5.    Educational  Objectives:    Realistic  and 
measurable  objectives  do  not  appear  as 
integral  parts  of  educational  decision** 
making.    They  tend  to  be  too  general 
or  too  specific,  are  not  relevant  to 
the  real  needs  of  the  organization, 
and  progress  toward  them  cannot  be 
easily  measured,  if  at  all* 


The  AMA  Approach  to  Management  Development 

In  order  to  solve  these  management  problems  in 
education  the  AI1A  attempts  to  improve  organizational  effec- 
tiveness through  the  development  of  individual  managers. 
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Emphasizing  "leadership  by  example"    the  AIIA  concentrates 
on  top  management  and  on  people  directly  responsible  to 
them.    Unless  top  management  is  attuned  to  the  principles 
of  good  management/  openly  supporting  and  practicing  them^ 
there  is  little  likelihood  that  overall  organizational 
effectiveness  will  increase.    The  AMA's  general  xrule  of 
thumb/  therefore/  is  to  start  at  the  top  of  the  hierarchy/ 
or  as  close  to  the  top  as  possible/  for  that  is  where  the 
power  to  change  an  organization  usually  lies. 

Within  this  top  group/  a  team  planning  process  is 
attempted.    Organizations  involved  in  the  training  actually 
develop  a  plan  for  their  organization.    It  is  argued  that 
by  experiencing  this  process/  the  participants  are  forced 
to  deal  with  their  management  problems  and  develop  new 
approaches  to  them.    If/  for  example/  decisionmaking  is 
clearly  overcentralized  and  individuals  within  the  organi- 
zation do  not  thoroughly  understand  their  responsibilities/ 
solutions  to  these  problems  are  developed.    In  this  case, 
the  team  may  decide  that  de-^isions  should  be  made  by  people 
who  are  responsible  for  them  and  set  an  objective  to  develop 
job  descriptions  and  performance  standards  that  clarify 
responsibilities . 

The  word  team  is  crucial  to  an  understanding  of  the 
AHA  approach  to  organizational  management  and  planning. 
AMA  believes  that  "the  crucial  factor  is  the  degree  to 
which  the  training/learning  experience  represents  a  change 
in  accepted  or  traditional  practices  and  procedures."'  If 
the  modified  behavior  does  not  radically  depart  from  past 
modes  of  administrative  operations  and  interpersonal 
relations/  then  its  adoption  should  proceed  smoothly.  But 
in  those  cases  where  modified  behavior  is  radically  differ- 
ent/ forces  within  the  organization  will  resist  and  may 
oppose  its  adoption  because  the  behavior  threatens  their 
accepted  ways.    Within  such  a  hostile  environment/  individ- 
uals may  find  it  difficult  to  retain  their  modified  behavior 
patterns  and  may  return  to  other  behaviors  more  in  line  with 
the  organization <^ 

Accordingly/  the  2U4A  seeks  to  change  the  behaviors  of 
entire  teams  of  managers.    By  involving  the  entire  e:tecutive 
staffs  of  the  experimental  educational  agencies/  AMA 
theorizes/  people  receive  suitable  reinforcement  from  their 
colleagues  for  maintaining  and  actually  diffusing  their  new 
approach  to  management  ax  1  planning  in  the  face  of  normal 
organizational  resistance.    Since  AMA  teams  constitute  the 
top  management  of  the  organization/  this  mutual  reinforce- 
ment is  expected  to  sustain  program  effects  as  they  percolate 
to  lower  levels. 8 


^Ibid./  p772. 


ERLC 


8cf.  Chapter  One  of  this  report  for  a  thorough  discus- 
f^.  sion  of  the  AMA's  actual  training  design  and  format. 
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The  AMA's  Pilot  Project 
The  two  major  objectives  of  the  orig?.nal  proposal 

were:^ 

1.  To  determine  the  feasibility  of 
developing  cind  applying  particular 
learning  methods  and  modified  con- 
tents of  AMA's  management  development- 
programs,  which  would  be  considered 
effective  for  training  various  levels 
of  educational  administrators* 

2.  To  introduce  and  experimentally  con- 
duct these  educational  programs  for 
representative  multi-state /  multi- 
level groups  of  educational  adminis- 
trators over  a  period  of  one  year. 

With  the  concurrence  of  USOE,  the  A!1A  listed  fourteen 
criteria  on  which  the  training  program  should  be  evaluated: 

1.  an  agreed-upon  definition  of  the 
agency's  mission / 

2.  established  continuing  objectives  and 
planning  procedures  for  long-range 
achievement  of  the  institution's  missicni 

3.  identified  resources  and  constraints / 

4.  differentiated  between  where  the  insti- 
tution is  going  and  where  it  wants  to  go, 

5.  modif ie3*previously  established  objectives, 

6.  identified  and  analyzed  alternative 
courses  of  action, 

1.    determined  priorities / 

8.  made  strategic  action  assignments/ 

9.  defined  standards  of  performance  for 
key  administrators / 

10*    specified  task  completion  dates / 

11.  designed  supplementary  planning  efforts / 

12.  assigned  responsibilities  to  subordinate 
units / 

13.  designed  a  methodology  by  which  future 
performance  may  be  evaluated  in  relation 
to  the  performance  specified  in  the  plan^ 

14.  produced  and  are  implementing  a  long- 
range  strategic  plan. 

The  Mili  perspective  insists  that  these  are  not  abstract 
principles  but  concrete  realities  requiring  concrete 


AMA|  0£.  ext.,  p.  4;  Klawuhn  and  Basso^  ££•  cit . , 

p.  5. 
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decisions.    As  such,  their  development  during  A»l«  training 
substantially  contributes  to  improving  the  decisionmaking 
process.    Aim  believes  that  the  only  way  to  learn  is  by 
doing.    The  AliA  concludes: 

.  ,  .  that  the  criteria  were  bcsed  on 
the  expectation  that  educational  mana- 
gers would  acquire  decisionmaking  skills 
and  successfully  apply  them  to  their 
own  education  agencies.    A  second,  al- 
though not  so  obvious  conclusion,  is 
that  the  training  aspects  of  the  Team 
Planning  Process  .  .  .  were  definitely 
of  secondary  importance.    That  they  were 
necessary  precedent  is  certainly  to  the 
point,  especially  from  the  theoretical 
viewpoint;  but  these  would  be  reinforced 
by  actual  application:    by  making  deci- 
sions, by  creating  a  strategic  plan;  by 
identifying  resotirces,  etc.iO 

The  final  criterian  by  which  to  assess  program  effectiveness 
was  actual  progress  toward  the  objectives  enumerated  in  the 
plans  produced.    Unless  the  changed  behaviors  and  new 
Dlans  had  some  impact  on  pupils  and  other  clients  for  whom 
they  were  intended,  the  program  could  not  really  be  called 
a  success. 

Inasmuch  as  effectiveness  is  a  function 
of  the  results  achieved  for  the  resources 
consumed,  any  final  judgment  as  to  the 
adaptability  of  business  management 
principles  to  education  must  be  reserved 
until  a  comparison  of  past  results  with 
future  results  as  expressed  in  each  in- 
stitution's plan  is  possible. 


Linking  Program  with  Organizational  Impact 

As  in  the  Yl  evaluation  we  have  concept  alized  the 
research  variables  used  in  this  project  in  terms  of  Likert  s 
causal,  intervening,  and  end-result  variables.-^''    Each  of 
the  next  three  chapters  will  treat  each  of  these  categories 
in  turn. 

The  causal  variables  are  discussed  in  Chapter  Four, 
Tl>a  Organizational  Planning  Process.  Causal  variables  are 
defined  here  as  "independent  variables  which  determine  the 
course  of  developments  within  an  organization  and  the 


O  ^°Ibid.  ^Wd.,  p.  ii. 

ERJC  -  ^Rensis  Likert,  The  Human  Orcranization  (New  York, 

1967)  ,  pp.  26,  29. 
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results  achieved  by  the  organization."       In  the  context 
of  this  evaluation  we  have  defined  the  AMA's  basic  train- 
ing goals  listed  above  as  being  causal  variables;  the 
extent  to  which  they  have  been  achieved  will  highly  deter- 
mine the  effectiveness  of  the  training  program  anr*  its 
consequent  impact  on  organizational  output.  ^  ^blish- 

ment  of  these  variables  is  seen  as  essential        he  develop- 
ment of  a  viable  planning  process  within  the  experimental 
organizations. 

The  intervening  variables  are  displayed  in  Chapter 
Five,  The  Development  of  the  Organizational  Management 
Environment.    These  describe  the  health  of  an  organization — 
in  this  case  the  internal  environment  of  the  organizations 
hale  against  the  professional  management  criteria  of  the 
AMA.    As  has  been  discussed  above,  the  AMA  has  specific 
process  objectives  for  leadership  style,  decisionmaking 
process,  and  management  team  relations  in  any  organization. 
It  is  our  purpose  to  evaluate  the  esctent  to  which  these 
organizational  processes  changed  as  a  result  of  the  train- 
ing program. 

Chapter  Six  is  concerned  with  the  evaluation  of  the 
effects  of  the  training  program  on  the  output  of  the 
organization.    We  have  defined  as  "md-result  variables  the 
plans  produced  by  the  organizations  and  measured  progress 
toward  the  achievement  of  the  objectives  specified.  Both 
aire  considered  equally  important.    An  observable  impact 
on  the  output  of  the  organizations  trained  ii  seen  as  the 
essential  precondition  of  a  positive  evaluation  of  the 
AHA's  pilot  project. 


Reporting  Format 

In  the  following  three  chapters  we  examine  as 
thoroughly  as  possible  the  attitudes  and  actions  taken  by 
tha  organizational  participants  concerning  the  various 
research  variables. 

The  analysis  of  orgcinizational  documents  appears 
first.    Most  of  this  material  is  discussed  in  Chapters 
Pour  and  Six  and  relates  specifically  to  the  plans  which 
have  or  have  not  been  produced  and  implemented  in  the 
experimental  organizations.    We  also  discuss  other  elements 
of  the  AMA  training  goals,  for  example,  performance 
standards  and  evaluation  methodologies  developed. 

Following  the  analysis  of  organizational  documents, 
data  concerning  our  attitudinal  research  variables  is 
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offered «    This  data  is  organized  so  that  the  interview 
conter. ^         ysis  appears  first.    These  data  emerge  from 
interv^L-'/ys  the  top  twelve  administrators  in  each 

experimental  state  and  in  the  original  control-state. 
Similar  interviews  were  not  conducted  in  the  Local 
Educational  Agencies,  so  this  analysis  speaks  only  of 
SED's. 

The  questionnaire  data  for  the  larger  populations 
in  the  states  is  presented  next.    As  the  reader  will 
recall  from  Chapter  Three,  questionnaires  were  completed 
by  the  twenty-four  people  who  completed  training  and  by 
two  subordinates  of  each  of  these  persons.    This  created 
a  total  population  of  about  72  in  each  state.    The  same 
procedure  was  followed  in  the  Control  States. 

Next  we  present  data  collected  in  the  Local  Educa- 
tional Agencies »    Because  of  the  wider  range  of  data 
available,  our  primary  focus  rests  on  the  State  Educational 
Departments.    In  each  chapter  SED  data  will  be  presented 
first,  followed  by  a  separate  section  on  the  LEA*s. 

In  examining  the  impact  on  the  SED^s  and  LEA's  we 
have  integrated  wherever  possible  the  data  developed  over 
the  two  year  life  of  this  evaluation.    Each  chapter  is 
followed  by  a  summary  in  which  specific,  data-based 
conclusions  are  made.    A  final  conclusion  and  judgment 
regarding  the  overall  effectiveness  of  the  American 
Management  Association's  pilot  project  appears  in  Chapter 
Seven. 
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CHAPTER  FOUR 


THE  ORGANIZATIONAL  PLANNING  PROCESS 


In  this  chapter  we  examine  thirteen  of  the  criteria 
put  forward  by  the  AMA  as  the  basis  of  training  program 
evaluation.^  These  "causal"  variables  are  independent 
varieJDles  which  should  influence,  at  least  theoretically, 
the  course  of  future  developments  within  the  participating 
educational  agencies  and  the  results  they  achieved. 2 
Achievement  of  these  criteria  alone  will  not  guarantee 
success  to  the  program;  much  depends  on  the  internal 
climate  of  the  organization  (intervening  variables)  and 
an  assessment  of  educational  gains  made  by  these  agencies 
due  to  training  (end-result  variables).    He  include  here 
evidence  of  the  AMA's  fourteenth  training  goal,  the 
implementation  of  a  long-range  strategic  plan.    But  satis- 
factory progress  on  the  causal  variables  is  a  first  step 
toward,  and  a  necessary  precondition  of,  progress  on  the 
other  variables. 

Specifically,  we  evaluate  the  criterion-referenced 
actions  taken  during  and  after  ATIA  training /  as  well  as 
participant  attitudes  toward    them.    Our  examination  is 
divided  into  four,  parts. 

The  first  area  (Organizational  Planning  Process  - 
Area  I)  deals  with  the  7  criteria  most  fully  developed  by 
the  experimental  units  during  the  training  program.  The 
??econd  area  (Organizational  Planning  Process  -  Area  II) 
considers  those  criteria  which  were  not  addressed  until 
after  the  training  program.    Although  the  AI^Ui  initially 
implied  that  all  criteria  would  be  met  to  some  degree  at 
the  training  site,  some  were  reserved  for  attention  only 
after  training  ended.    In  both  parts,  we  assess  v;hat  was 
accomplished,  offer  data  displays  based  on  participant 
attitudes,  and  analyze  why  certain  things  happened  and 
others  did  not. 

In  the  third  area  (LEA  Data) ,  data  gathered  xrom 
participating  local  .education  agencies  (LEA's)  is  intro- 
duced.   Here  we  study  the  attitudes  expressed  by  school 
district  personnel  toward  selected  criteria.    A  two-year 


^American  Management  Association,  "Feasibility  and 
Pilot  Programs  Proposal^:    Adapting  and  Testing  Business  Man- 
agement Development  Programs  for  Educational  Administrators" 
PP^^.  (June  22#  1970),  pp.  4-5  (mimG;ograph) . 

HsMsa  ^Rensis  Likert,  The  Human  Organization  (New  York: 
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data  display  similar  to. that  used  for  the  SED's  is  pre- 
sented.   Finally^  we  offer  some  conclusions  about  the 
attitudinal  changes  produced  by  AHA  training  in  the  LEA's 
in  relation  to  the  criteria. 

The  fourth  area  (Overall  Conclusions  and  Summary) 
assesses  ana  summarizes  the  preceding  three  sections.  It 
is  an  overall  outline  of  the  extent  to  which  wc  believe 
the  causal  variables/criteria  have  been  mot  by  AMA  training 
in  the  experimental  educational  agencies.    The  concluding 
section  is  based  not  only  on  whether  or  not  each  variable/ 
criterion  was  accomplished  but  also  on  how  participants 
view  them*    As  we  have  urged  in  previous  chapters,  actions 
and  attitudes  are  mutually  dependent;  vrithout  changes  in 
both/  prospects  of  long-terra  organizational  change  are 
diminished. 


AREA  I:     THE  CAUSAL  VARIABLES  1  THROUGH  7;  SED'S 

Here  we  examine  the  extent  to  which  the  first  seven 
goals  of  the  AMA  were  accomplished  and  what  the  participants 
thought  of  them.    The  following  discussion  is  divided  into 
two  parts.    The  first  part  broadly  discusses  expected 
results/  actual  results,  and  significant  issues  raised  by 
these  items.    The  second  part  displays  and  discusses  the 
attitudinal  data. 


Section  1:  Action 


A.    Expected  Results 

We  will  enumerate  the  first  seven  AMA  criteria,  and 
provide  definitions  where  appropriate.    The  definitions 
paraphrase  those  used  by  AHA  in  the  planning  documents. 

1.    Agreed  upon  a  definition  of  the  institution's 
mission; 

Mission;    The  broadest,  most  comprehensive 
statement  that  can  be  made  about  central  or 
continuing  purpose.    The  chief  function  or 
responsibility  of  an  organization  which 
justifies  continuing  support  of  the  organi- 
zation and  provides  initial  direction  for 
its  management  or  administration.  The 
purpose  of  the  mission  statement  is  to 
provide  clear  focus  for  the  resources  of 
the  organization. 


McGraw-Hill  Book  Company,  1967),  pp.  26,  29. 


95 


't 

1 


Established  continuing  objectives  and  planning 
procedures  for  long  range  achievement  of  the 
institution's  mission; 

Continuing  Objectives;    Statements  of 
general  direction  or  intent.    A  continuing 
objective  is  broad,  timeless  and  unconcerned 
with  particular  achievement  within  a  speci* 
fied  period. 

Identified  resources  and  constraints; 

Resources:    An  estimate  of  the  personnel # 
money,  material,  and  information  available 
to  the  institution  to  do  what  it  wants  to  do. 

Constraints:    Internal  or  external  forces 
which  influence,  impede  or  prevent  the  insti-* 
tution  from  doing  what  it  wants  to  do. 

Differentiated  between  where  the  institution  is 
going  and  where  it  wants  to  go; 

Modified  previously  est  abolished  objectives; 

Specific  Objectives:    Are  quantifiable  and/or 
observcible  achievements  which  can  be  measured 
within  a  given  time  and  under  specified  con- 
ditions.   Specific  objectives  whould  reflect 
the  critical  factors  required  for  the  attain-- 
ment  of  a  continuing  objective.    The  achieve- 
ment of  a  specific  objective,  therefore, 
contributes  toward  the  attainment  of  the 
overall  goal.    Objectives  should  be  clearly 
differentiated  from  the  means  (strategies) 
employed  to  attain  them. 

Identified  and  analyzt>d  alternative  courses  of 
action; 

Strategies;    Are  the  programmatic  means  used 
to  attain  a  specific  objective? 

Determined  priorities; 

Priorities;    Judgment  of  relative  importance 
of  objectives  when  considered  in  relation  to 
significant  criteria.    Objectives  with  "high" 
priority  are  emphasized  more  than,  cuid  usually 
implemented  prior  to,  those  with  "low" 
priority. 
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B.    Actual  Results 

T€iking  the  above  seven  causal  variables/criteria, 
we  examine  how  much  they  have  been  accomplished,  at  least  on 
paper.    We  drew  no  conclusions  hero  as  to  their  relevance  . 
to  actual  organizational  behavior*    In  the  next  section  we 
offer  supplementary  evidence  based  on  participant  attitudes. 
Our  present  purpose  is  merely  to  determine  the  extent  to 
which  the  State  Education     Departments  performed  the  above 
exercises  as  stipulated  in  the  original  project  proposal. 
The  following  chart  summarizes  our  findings /  and  all  these 
items  are  explained  in  the  following  text. 

SUMMARY  OP  ACTION  FINDINGS 
SED  LEVEL 
AMA  CRITERIA  1-7 
PALL,  1970  to  SPRING,  1972 

 Amount  of  Progress  


M£i  Criteria  Minimum     Moderate  Maximum 

1.  Agreed  upon  a  definition 
of  the  institution's 

mission  El^  E2 

2.  Established  continuing 
objectives  and  planning 
procedures  for  long-- 
ramge  achievement  of 
the  institution's 

mission  El,  E2 

3.  Identified  resources 

and  constraints  El,  E2 

.  4.  Differentiated  between 
where  the  institution  is 
going  and  where  it  wants 

to  go  El,  E2 

5.  Modified  previously 

established  objectives  El,  E2 

6.  Identified  and  analyzed 
alternative  courses  of 

action  El,  E2 

7.  Determined  priorities  El  E2 
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Both  experimental  states  had  similar  experiences 
and  similar  results  while  at  Hamilton.    Each  underwent 
training  and  prepared  materials  covering  all  seven  areas. 
There  were  some  differences  in  the  form  of  presentation  as 
well  as  in  content  but  these  arc  not  significant.    By  and 
large,  comparable  degrees  of  progress  were  made  by  the  two 
groups  at  the  training  site. 

A  mission  statement  and  set  of  continuing  objectives 
were  developed  by  ESED  #1  and  #2.-^    The  planning  documents 
devoted  considerable  space  to  an  external  and  internal 


These  can  be  found  in  the  planning  documents  pro- 
duced by  both  states.    See:    Raymond  E.  Klawuhn  and  Alexander 
J.  Basso,  "Final  Report:    Adapting  and  Testing  Business 
Management  Development  Programs  for  Educational  Administra- 
tors'^  (January,  1972),  ESED  #1:  Vol.  Ill  &  IV;ESED  #2:  Vol.  II 

Examples: 

ESED  #1  ESED  #2 

Mission  Statement 


The  mission  of  the  .  .  . 
state  education  agency  is 
to  ensure  through  informed 
and  effective  leadership  at 
the  state  and  local  levels 
those  learning  experiences 
which  are  compatible  with 
individual  needs,  interests, 
and  capabilities  and  which 
will  lead  to  continued 
education  and/or  employment 
for  all  students. 


The  mission  of  the  .  .  . 
state  department  of  educa- 
tion is  to  ensure  that  the 
current  and  continuing 
educational  needs  of  the 
children,  youth,  and  adults 
of  the  state  are  met  compre- 
hensively, effectively,  and 
efficiently. 


Continuing  Objectives 


Consistent  with  a  realistic 
appraisal  of  their  needs, 
interests,  and  abilities, 
all  students  in  the  state 
of  .  .  .  leaving  the 
elementary-secondary  schools 
will: 

1.  Be  qualified  to  either 
continue  formal  educa- 
tion or  become  employed 

2.  Demonstrate  competencies 


To  insure  that  each  student 
completing  his  elementary- 
secondary  school  program: 

1.  Is  prepared  to  continue 
his  education  or  to  meet 
the  requirements  of  the 
job  market  in  a  field 
consistent  with  his  inter 
est  and  ability. 

2.  Has  a  command  of  the 
learning  skills.  •  •  • 
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analysis  of  both  organizations  including  a  review  of 
resources  and  constraints.^    These  are  the  areas  of  greatest 
training  site  accomplishment . 

Neither  group  was  able  to  complete  action  on  every 
criterion  as  originally  intended.    Less  was  immediately 
accomplished  on  some  than  on  others,    while  good  starts 
were  made  on  enumerating  specific  objectives  and  strategies ^ 
this  catalogue  was  not  completed  at  Hamilton.    In  addition, 
ESED  #1  did  not  finalize  its  priorities  until  later. ^ 


in  the  arts  sufficient  to 
enable  the  student  to  make 
wise  value  judgments  and 
to  make  creative  use  of 
his  artistic  talents.  .  .  . 

4 

Internal  Analysis:    A  catalog  of  factors  which  col- 
lectively describe  the  nature  of  the  institution,  its  capabil- 
ity and  limitations;  this  analysis  is  to  be  restricted  to 
those  factors  which  are  within  the  control  of  the  institution 
and  which  play  a  significant  role  in  determining  the  most 
appropriate  course  of  action  for  the  institution.  Topics 
considered  included:    organization,  beliefs,  characteristics, 
functions,  resources,  strengths,  and  weaknesses. 

External  Analysis:    A  catalog  and  analysis  of  those 
factors,  outside  of  the  control  of  the  organization,  which 
serve  as  constraints  or  whose  interaction  with  the  organiza- 
tion determine  the  appropriate  behavior  modes  for  the  organi- 
zation.   For  each  of  the  critical  factors  identified  the 
team  made  explicit  assumptions  describing  expected  trends 
in  each  of  the  areas  for  the  planning  period.    While  these 
factors  aire  beyond  the  control  of  the  organization  there 
should  be  a  common  understanding  of  the  trends,  rate  of 
change,  and  kind  of  change  anticipated  in  each  of  the  areas. 
This  will  insure  that  all  plans  will  be  based  on  the  same 
assumptions  about  the  future.     (AMA  definitions.) 

A  recurring  problem  in  the  development  of  plans  accord- 
ing to  the  AMA  process  is  the  absence  of  any  direct  contact 
between  the  external  and  internal  analyses  and  the  plans  sub- 
sequently developed,    while  listing  beliefs,  assumptions, 
resources,  etc.  probably  serves  a  useful  "consciousness 
raising"  function  by  keeping  these  considerations  before 
participants,  the  extent  to  which  plans  reflected  them  is 
questionable. 

5 

The  subject  matter  priorities  in  ESED  #1  are  early 
childhood  education,  career  education,  human  relations,  and 
reading.    ESED  #2  has  established  early  childhood  education, 
human  relations  and  reading,    of  the  two  states,  the  priorities 
in  ESED  #2  are  more  obvious  and  publicized  at  this  point.  See: 
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Reasons  given  for  these  postponements  included  lack 
of  time  and  qualified  personnel  and  an  underestimation  of 
how  long  it  would  actually  take  to  lay  out  an  action  plan 
for  an  entire  state  education  deoartment.    According  to 
some  persons  with  whom  we  talked',  delay  also  served  to 
avoid  conflict,  especially  over  the  priorities. 

V^e•rG  just  getting  around  to  doing  that 
now;  giving  priority  to  some  divisions 
means  you  have  to  downgrade  others.  No 
one  around  here  wants  to  get  caught  in 
that  crossfire.    When  they  do  announce 
the  winners  and  losers,  they'll  probably 
do  it  before  dawn  one  day  emd  then  duck 
to  avoid  getting  their  heads  blown  off. 6 

This  deferment  was  not  permanent.     In  terras  of  setting 
things  down  on  paper  and  formally  agreeing  to  them,  action 
had  been  taken  on  all  criteria  once  the  first  phase  of  train- 
ing ended  and  participants  returned  home. 

In  any  event,  what  really  natters  is  not  a  count  of 
the  number  of  objectives,  strategies  and  studios  written 
(either  at  Hamilton  or  back  in  the  organizational  environ- 
ment) but  an  analysis  of  what  has  happened  to  them  in  the 
year  since  training  ended.    The  mission  statement  and 
continuing  objectives  are  essentially  the  same  today  as 
they  were  immediately  after  training.    However,  the  specific 
objectives  and  strategies  have  been  extensively  revised  for 
reasons  that  will  become  clear  below.    Typical  of  both 
states  was  the  comment  that: 


State  Superintendent,  "Memo  to  Professional  Personnel: 
Strengthening  the  Dopartmont"  (November  3,  1971);  in  addi- 
tion, the  research  toan  has  examined  a  series  of  pamphlets 
cn  educational  priorities  which  has  been  circulated  to 
external  groups  and  educational  agencies. 

Sxecutive  Staff,  ESED  fil 

To  protect  the  anonymity  of  interviewees  but  orovide  the 
reader  with  some  information  on  their  hierarchical  raiik  and 
agency,  the  following  identification  system  will  be  used 
throughout  the  remainder  of  this  report: 

r'SED  #1  ESED  #2 

TOP  I'IANAGEHEHT  executive  staff  executive  staff 

MIDDLE  MAriAGE.'ffiNT      division  director       division  director 

PROFESSIONAL  STAFF    consultant/special-  consultant 

ist 
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They  have  been  revised.    The  majority 
of  the  specific  objectives  have  been 
discarded  to  bo  redeveloped  by  persons 
with  competence  in  those  areas  and  worth 
appropriate  input  from  persons  to  be 
affected.    So  there's  no  question  but 
that  they  will  be  revised  and  will  come 
out  in  a  somewhat  different  form. 7 

Specific  objectives  have  been  extensively  revised 
by  individuals  within  the  SED's  who  have  the  competence  and 
responsibility  for  the  specific  areas  in  question.  AI4A 
standards  of  what  defines  a  good  objective  and  other 
elements  of  the  overall  planning  process  have  been  retained 
in  both  states.    Thus,  the  content  has  been  changed  in  some 
areas,  thought  the  Mik  structure  has  been  retained. 

In  dealing  with  outside  constituencies,  some  changes 
have  had  to  be  made,  particularly  in  ESED  #2;  they  had 
trouble  selling  thciir  ideas  to  the  State  Board  of  Education. 
The  State  Board  found  it  difficult  to  understand  and  accept 
the  plans  developed  subsequent  to  AMA  training.  Except 
for  internal  purposes  the  department  had  to  jettison  the 
AMA  terminology  ("mission,"  "continuing  objectives,"  etc.). 

This  is  another  thing  that  I  think 
wasn't  adequately  explored  in  the  train- 
ing with  AI!A.    Your  presentation  to 
different  audiences,  I  think,  almost  has 
to  vary.    Do  you  understand  what  I  mean 
by  that?    I'm  sure  that  you  do  this  as 
you  speak  to  groups.    You  won't  try  to 
snow  a  PTA  group  with  the  same  kind  of 
language  that  you  might  luse  withl  a 
graduate  class.    I  think  we  learned  a 
valuable  lesson,    within  the  department, 
within  the  executive  staff  of  the  depart- 
ment at  least,  we  can  use  a  certain  kind 
of  terminology.    You  can  oven  do  it  with 
the  entire  professional  staff  to  some 
extent  -  so  long  as  you  exercise  some 
care.    To  a  lay  board,  your  approach  has 
to  be  .  .  .  different.    You  don't  have 


Executive  staff,  ESED  #2.    See  also:    A  comparison 
of  the  planning  documents  produced  during  AHA  training  and 
the  latest  plans  of  the  various  operating  divisions  in  both 
states  made  available  to  the  research  team. 
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to  go  in  and  say  "This  is  a  continuing 
objective  and  this  is  a  specific  objec- 
tive and  this  is  a  strategy*"    You  say 
"This  is  a  major  goal;  or,  this  is  ope 
of  the  particular  goal  areas  we  want  to 
work  in  and  here  are  some  of  the  activi- 
ties associated  with  it*"    That's  far 
more  understandable;  they  don't  feel 
like  they're  being  snowed  vrith  a  lot  of 
jargon,  when  you  approach  it  in  this 
way.  ^ 

In  terras  of  the  first  seven  AMA  criteria,  the  "plan- 
ning process"  first  introduced  to  the  experimental  states 
at  Hamilton  has  proved  more  lasting  than  the  form  or  content 
of  the  plans  themselves.    This  is  less  true  of  ESED  #1 
than  #2.    Developing  them  was  a  real  learning  experience 
for  most  participants,  although  the  extent  to  which  the 
plans  represented  hard  decisions  and  viable  organizational 
documents  remains  questionable. 


C.    Emerging  Issues 

Some  of  the  questions  raised  by  participants  about 
this  part  of         training  are  highlighted  in  the  following 
material,  which  provide  some  basis  for  discussion  of  the 


Executive  Staff,  ESED  #2.    Sec  also,  "Goals  for 
[ESED  #21  Supporting  Services,"  which  are  the  revised  plan 
ning  documents  presented  to  the  state  board  after  the 
initial  adverse  reaction.    Instead  of  breaking  out  plans 
into  mission  statement,  continuing  objectives,  specific 
objectives  and  other  elements  of  the  AMA  planning  process, 
they  presented  the  state  board  with  this  format: 

Goals  for  [ESED  #2]  Supporti  ig  Services 

Goal  Area:  Priority  Level: 

Justification: 

Problem: 

Present  Status: 

Results  Desired: 

Plan  of  Action: 

This  outline  is  not  constant  throughout  all  goal  areas  but 
the  above  headings  appear  in  most  of  the  plans  made  avail- 
able to  the  research  team. 
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program,  possible  changes  which  nay  be  considered  now  or 
proposed  in  the  future,  and  reasons  for  the  attitudes 
expressed  in  the  next  section. ^ 

1.    Baseline  Data 

One  of  the  reasons  why  more  specific  performance 
criteria  were  not  developed  at  Kamilton  wr.s  that  the 
states  entered  training  mostly  without  data  applicablt;  to 
the  Planning  Process.    To  be  sure,  their  expert  judgmunt 
was  sufficient  for  the  statement  of  generalized  mission 
and  continuing  objectives.    However,  the  state  executives 
were  not  prepared  to  develop  behavioral  specific  objectives 
and  strategies  that  demand  precise  percentage  targets  and 
deadlines. 

For  example,  not  a  few  thought  it  rather  presumptuous 
to  state  that  "By  1975,  85%  of  all  11  year  old  students 
v/ill  be  able  to  read  as  determined  by  appropriate  criterion 
referenced  tests"  v/hen  no  baseline  data  existed  to  show  how 
many  currently  read  at  that  level.    The  intersession  neriod 
was  not  sufficiently  long  to  gather  this  information."  In 
fact,  some  degree  of  uncertainty  still  prevailed  in  both 
states  six  months  after  training  ended  (Kov.  1971);  the 
statewide  assessment  programs  unden/ay  in  both  experimental 
states  only  began  to  bear  fruit  recently. 

The  absence  of  supporting  data  raises  some  questions 
as  to  the  reality  on  which  the  plans  were  based.    This  was 
made  clear  to  ESED  ^2  by  one  of  the  State  Board  members: 

One  of  the  things  said  by  a  board  mem- 
ber, which  I  thought  was  beautiful, 
concerned  the  reading  goals  presented 
to  him.    I  forget  what  it  [the  goal] 
said  ...  80%  of  all  15  year  olds  or 
12  year  olds  or  something  like  this,  or 
maybe  it  said  B5K .    VTiatever  it  said, 
his  response  was  "I  can't  accept  that. 
You  have  no  basis  for  establishing 
percentages,  no  baseline  data.    For  all 
I  know  95%  of  them  might  be  able  to 
do  that  now  and  I'd  be  saying  I'd  settle 
for  a  10%  drop."    He  was  right;  we  had  no  ,^ 
backup  data  to  supnort  our  recommendations. 


Selections  from  p^srsonal  testimony  have  already 
been  used  as  an  analytical  device  in  rSED  ^2  -  excerpts 
from  LEA  reaction  to  preliminary  SED  plans  have  been  written 
up  and  made  available  to  the  research  team  and,  we  assume, 
to  other  interested  individuals  as  well. 

<^  10 
ERXC  Executive  Staff,  ESED  #2. 
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2.    Student  Behavioral  Objectives 

An  integral  part  of  the  AMA  approach  to  educational 
planning  is  to  focus  on  changes  in  student  learning. 
Objectives  must  be  stated  in  terms  of  student  behavior  for 
this  is  the  end  result  the  organization  seeks.    This  in 
the  AMA  view  avoids  diffusing  planning  efforts  on  manage- 
ment concerns  which  are^  after  all,  means  to  student 
achievement  cognitively  and  affectively. 

This  exclusive  concentration  on  student  behavioral 
objectives  has  created  several  problems  in  the  experimental 
agencies  involved  in  this  study.    This^  in  part,  may  be 
due  to  a  poor  understanding  of  the  central  concepts  on  the 
part  of  the  individuals  who  were  trained. 

First/  one  of  the  most  frequent  comments  heard  on 
the  state  level  concerns  the  frustration  participants  felt 
as  they  tried  to  state  objectives  in  student  behavioral 
terms.    They  argue  that  within  the  educational  system  the 
State  Department  of  Education  can  deal  with  students  only 
through  the  LEA's.    It  is  the  LEA's,  the  argument  goes, 
who  determine  in  the  final  analysis  what  will  or  will  not 
happen  to  students.    ThuS/  forcing  SED  personnel  to  write 
objectives  in  student  behavioral  terms  has  resulted  in  some 
resistance  within  the  SED's  involved. 

Yet  the  biggest  difficulty  we're  going 
to  have  and  the  hangup  that  AMA  has  not 
solved  and  we  haven't  solved  is  how  you 
measure  behavioral  change  in  students 
by  objectives  that  have  been  set  by  a 
state  agency  which  in  reality  has 
no  direot  input  to  children.    We  haven't 
solved  that  problem.    Thus/  when  we  say 
write  objectives  in  behavioral  terms  of 
what's  going  to  happen  to  kids  and  then 
you  ask  the  question  of  how  effective 
have  our  objectives  been,  to  what  extent 
have  they  been  reached ,  we'll  never 
answer  it  because  we  are  not  working 
directly  with  children*  .   .   .  And  I 
don't  know  how  to  settle  this.   .  .  . 
I'm  not  sure  I'll  ever  know.     But  I'll 
tell  you  one  thing,  if  the  insistence 
on  writing  objectives  this  way  was  toned 
down  you  would  win  the  support  of  most 
of  the  consultants  overnight  for  the 
planning  process. 


Executive  Staff,  ESED  #1. 
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I  think  we  tried  to  reach  a  level  of 
specificity  which  was  unrealistic  for 
our  organization,  the  State  Deoartment 
of  Education.    I  think  it  would  have 
been  far  more  realistic  for  a  classroom 
organization,  rftther  than  for  a  state 
department  of  education. 12 

On  the  other  hand,  some  SED  personnel  think  it  quite 
appropriate  for  objectives  to  be  expressed  in  terms  of 
student  behavior.    While  sensing  the  intermediary  role  of 
LEA'S,  they  are  still  convinced  that  all  educational  . 
personnel  must  think  primarily  in  terms  of  how  programs 
affect  student  learning.    All  else  merely  supports  these 
larger  ends. 

We  have  an  effect.    The  LEA' a  have  an 
effect.    But  the  name  of  the  game  is 
how  both  of  us  can  do  a  better  job  for 
students.    Concentrating  our  attention 
here  only  on  helping  LEA's  help  students 
would  abdicate  our  responsibility  to 
provide  statewide  coordination  of  the 
education  of  our  young  people. ^3 

Second,  the  emphasis  on  student  behavioral  change 
as  the  primary  content  of  objectives  does  render  supportive 
services  (evaluation,  research,  budgeting,  etc.)  peripheral. 
In  ESED  #2,  these  staff  offices  were  discouraged  from 
developing  their  own  specific  objectives  and  relegated  to 
"sustaining  strategies."    While  this  may  seem  a  small 
sacrifice  to  make  in  order  to  permit  full  concentration 
on  students  as  clients,  it  was  the  source  of  some  resent- 
ment.   It  was  eventually  realized  that,  although  the 
exact  form  may  differ,  staff  roles  require  the  same 
planning  and  goal-setting  as  their  line  counterparts. 
This  planning  "discrimination"  is  now  scheduled  for 
termination.    "I'm  about  ready  to  admit,"  stated  one 
division  director,  "that  if  the  guy  in  the  Finance  Depart- 
ment wants  an  objective  relating  to  how  he  keeps  his  books, 
I'm  just  as  happy  calling  that  an  objective  as  I  am  a 
strategy  of  the  Department. "14 

The  fact  that  the  development  of  such  objectives 
in  student  behavioral  terms  was" never  the  approach  taken 


Executive  Staff,  ESED  #2. 
Consultant,  ESED  #1. 
Executive  staff,  ESED  ff2. 


105 


by  the  AtlA  would  seon  to  indicate  a  serious  misunderstand- 
ing of  the  concepts  involved  on  the  part  of  some  partici- 
pants in  the  training. 

Third,  the  use  of  percentage  figures  and  deadlines 
raised  some  philosophical  questions.    Some  SED  personnel 
wondered  whether  the  department  was  concentrating  only  on 
objectives  which  could  be  measured  while  ignoring  those 
which  could  not.    There  was  special  concern  about  the 
affective  domain.    One  person  was  reminded  of  an  Orwellian 
experience  in  which  "Big  Brother"  uould  ordain  that  children 
should  learn  X  amount  by  Y  date.    How  much  of  this  objec 
tion  stems  from  the  relative  modernity  and  special  jargon 
of  the  behavioral  approach  to  instruction  is  difficult 
to  discern. 


3#    Selection  Process 

As  mentioned  previous  ly,  part  of  thn  i\IL'^.  approach 
is  to  initiate  training  with  top  management.    By  starting 
at  the  top  of  the  hierarchy/  they  hope  to  gain  the  support 
of  the  power  liolders  of  the  organization  and  to  encourage 
program  implementation  through  their  subordinates.    It  is 
also  believed  that  certain  planning  operations  (especially 
the  mission  statement  and  continuing  objectives)  need  the 
broad  perspective  of  those  who  oversee  the  whole  structure. 
While  this  approach  may  have  suited  business  clients 
without  difficulty,  ropeating  it  in  the  two  ex;  rimental 
state  educational  agencies  raised  questions. 

Taking  only  the  top  twelve  managers  (or  Executive 
Staff)  for  agency-wide  planning  (despite  the  intersession 
and  later  involvement  of  the  program  services  people) 
excluded  those  with  special  expertise  in  the  subject  areas 
covered  during  training. 

I  think  that  instead  of  our  executive 
staff  going  to  the  training  .  .  . ,  we 
would  have  been  well  advised  to  have 
some  of, our  talented  specialists  along 
with  us  to  give  us  some  -'^dvice  here  and 
there  and  to  clarify  the  issue  a  little 
more  than  it  was. 15 

This  did  not  become  apparent  until  the  specific  objective 
phase  was  reached  and  preliminary  generalizations  had  to 
give  way  to  the  specificity  of  deadlines/  performance 
figures/  and  evaluation  strategies.    Some  respondents 


Executive  Staff/  ESED  #2. 
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felt  that  they  were  throwing  around  ntimbers  and  dates 
without  the  required  background  knowledge. 

Others  objected  1;o  the  selection  process  on  differ- 
ent grounds.    While  they  conceded  that  selection  by 
hierarchical  level  might  be  satisfactory  for  corporations^ 
they  argued  that  "top"  in  public  agencies  is  more  complex 
than  in  private  business.    Public  agencies  are  open  to 
pressures  from  more  groups  at  more  levels  (e.g.,  legislature, 
governor's  office,  citizen  panels,  education  associations, 
etc.).    These  groups  often  have  mutually  exclusive  interests 
and  values;  occasionally  these  interests  are  antithetical 
to  those  of  the  State  Agency  and  the  AMA  program. 

Second,  by  not  directly  involving  State  Boeird  members 
(the  public  educational  equivalent  of  the  corporate  board 
of  directors),  subsequent  embodiment  of  some  aspects  of  AMA 
training  was  made  more  difficult.    Besides  the  lack  of 
understanding  of  crucial  AMA  pl£mning  concepts,  cinother 
factor  working  against  the  new  planning  effort  was  the 
unwillingness  of  sone  Board  members  to  commit  themselves 
to  any  long-term  plans?  they  felt  such  a  commitment  would 
mean  losing  their  legitimate  powers  of  review.  Involvement 
in  the  training  process  might  have  ameliorated  this  problem. 

I  think  it  [the  training  program]  would 
have  had  more  unity  to  it  if  the  board 
could  have  recognized  and  seen  our 
phraseology  as  a  good  method;  but,  it's 
easy  to  unders*:and  why  they  had  diffi- 
culty in  seeing  what  we  were  trying  to 
do.    None  of  them  went  to  AMA  with  us 
when  we  went  up  there. 


Legal  constraints  is  a  third  factor.    The  management 
of  the  educational  agencies  was  somev^hat  hamstrung  by  them, 
and  did  not  have  as  much  maneuverability  for  change  as  some 
would  have  liked.    It  was  suggested  that  some  of  their 
"controllers"  might  also  have  sat  in  on  the  training.  Most 
frequently  mentioned  as  those  whose  lack  of  understanding 
has  complicated  implementation  of  plans  has  been  the 
Controller's  Office  in  ESED  #1  and  the  state  personnel 
officer  in  ESED  #2. 


They've  got  rules  and  regulations  (I 
don't  J'now  where  in  the  hell  they  get 
the  rules  and  regulations  from) .  But, 


Executive  Staff,  ESED  #2.    One  board  member  from 
ESED  #1  did  attend  one  week  of  training  which  may  account 
to  some  extent  for  the  cooperation  of  the  board  in  that 
state. 
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we  know  that  if  wo  have  a  man  wc  know 
would  fit  a  position  and  he's  in  another 
area,  it's  hard  as  the  dovil  to  shift. 
The  organization  .  .  •  is  geared  to  the 
same  thing  every  year.    Maybe  doing  a 
poor  job  a  little  bit  better  or  maybe 
rendering  an  unneeded  service  a  little 
bit  better* 17 

Fourth/  the  negative  impact  of  having  top  management 
return  to  the  agency  with  a  plan  calling  more  for  subordinate 
consultation  than  participation  was  mentioned.  Despite 
the  seemingly  endless  round  of  meetings,  task  forces,  and 
study  groups/  a  number  of  personnel  felt  that  they  were 
being  asked  to  react  to  a  fait  accompli.    Others  questioned 
whether  support  for  the  plans  would  not  be  threatened  by 
asking  people  to  implement  them  who  did  not  directly  help  to 
develop  them.    "This  is  something  coming  down  from  the 
top/"  one  division  director  told  us#  "now  what  are  we 
supposed  to  do  with  it?" 18 


We  conclude  this  section  by  reiteratir.g  that  these 
were  frequent  criticisms  and  comments  cibout  those  aspects  of 
AiyiA  training  related  to  the  first  seven  evaluative  criteria. 
They  were  made  by  people  who  participated  in  the  progremi 
or  who  were  later  in  a  position  to  react  to  it.  Some 
criticisms  undoubtedly  were  prompted  by  the  usual  intra- 
bureaucratic  rivalries  and  reluctance  to  adapt  to  novelty. 
Others  may  have  resulted  from  a  lack  of  understanding  of 
the  planning  principles,  processes,  or  the  AflA's  adaptation 
of  them.    Nevertheless,  the  participants'  criticisms  cannot 
be  dismissed  on  these  grounds.    The  issues  were  raised 
often  enough  by  people  in  responsible  positions  to  justify 
at  least  passing  consideration. 


Section  II:    Attitudinal  Data 

As  indicated  above,  the  data  presented  in  this 
section  pertain  to  the  first  seven  training  goals  of  the 
AMA.    V?e  have  divided  this  section  into  two  sub-areas; 

A.  Definition  of  the  Mission  of  the  Organization 

B.  The  Development  of  Organizational  Objectives 
and  Priorities 


Executive  Staff,  ESED  #1. 
Division  Director,  ESED  #1. 
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Throughout  this  section  and  the  balance  of  the  report^  the 
content  analysis  data  will  be  presented  first.  This 
material  contains  the  reactions  of  the  Ton  Management  of 
the  two  experimental  organizations  and  the  first  control 
state  (CI).    Relevant  questionnaire  items  which  have  been  ^ 
drawn  from  multiple  levels  of  the  organizations  follow 
the  content  data. 

In  displaying  content  analysis  data^  four  comments 
are  provided. 

1.  Interview  Question:    Question  from  which  the 
data  is  drawn. 

2.  Range  of  Scale  Possibilities:    A  seven  point 
scale  was  used  for  all  interview  data.  Be- 
cause of  the  nature  of  the  interview  data  the 
terms  employed  along  this  scale  changed  from 
question  to  question.    In  all  cases/  however # 
the  intensity  direction  of  the  scales  was  the 
same/  i.e./  the  more  positive  the  response 
the  higher  it  was  rated  on  the  scale;  the 
less  positive  the  response  the  lower  it  was 
rated  on  the  scale. 

3.  Point  of  Time:    For  each  item  the  points  of 
time  in  which  the  question  was  asked  will  be 
indicated.     (Tl=Pall/  1970/  T2=Spring/  1971/ 
T3=5Pall/  1971/  T4«Spring/  1972.)     In  some 
cases  this  will  include  all  four  points  of 
time/  while  in  others  the  data  will  have  been 
gathered  in  Y2  only. 

4.  States:    We  identify  the  states  to  which  the 
category  applied. 

In  the  case  of  the  questionnaire  data  we  need  to  indicate 
only  t!ie  points  of  time  for  which  the  question  is  relevant. 
The  range  of  scale  oossibilities  v/as  the  same  in  all  cases/ 
(1)  not  at  all  to  (7)  very  often.    The  interview  questions 
were  asked  of  all  the  states.    As  indicated  above/  ques- 
tionnaire items  weie  asked  in  Control  State  *2  in  points 
of  time  T3  and  T4  only/  because  this  state  was  added  to 
the  research  design  in  Year  2. 

In  most  cases/  training  effects  were  determined  on 
the  basis  of  the  T1-T4  comparisons.    The  research  team  is 
most  confident  of  these  conclusions.    However/  due  to 
recent  additions  of  items  to  the  questionnaire  or  interview/ 
it  was  sometimes  necessary  to  assign  effects  after  a  T3-T4 
comparison  alone.    VJhile  the  absence  of  pre-training  data 
(or  in  some  cases  control  groups)  prompts  us  to  be  more 
cautious  in  our  interpretations/  we  are  quite  certain  our 

erJc 
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decisions  arc  valid.    This  is  particularly  true  where  wo 
obtained  organizational  documents  to  support  our  analysis 
(e.g.,  in  the  existence  of  performance  standards).  Further- 
more, the  T3-T4  comparisons  often  include  a  second  Control 
SED  vjhich  was  not  included  in  T1-T2  and  thus  could  not  be 
included  in  a  T1-T4  comparison.    The  presence  of  such 
multiple  measures  provides  double  coverage  on  many  items. 
For  these  reasons,  we  felt  justified  in  attributing  posi- 
tive effects,  no  effects,  cr  negative  effects  on  T3-T4  as 
well  as  T1-T4  comparisons.  ^ 

But  in  other  cases,  the  question  may  only  have  been 
asked  at  one  point  in  time  (T4).    This  is  insufficient  data 
on  which  to  base  a  judgment  of  training  effects,  cind  v;e 
made  no  such  judgments.    Eowever,  the  collected  data  will 
be  presented  to  give  us  a  better  overall  perspective. 
These  one-time-items  often  support  comments  made  earlier 
relating  to  actions  taken  after  training.    The  T4  state- 
ments from  which  data  was  derived  will  be  displayed  in  the 
data  stunmaries  of  the  effects  of  training  following  each 
area  to  review  precisely  what  items  were  considered  in 
that  section.    Nevertheless  no  judgment  is  made  of  what 
effects,  if  any,  the  ATIA  nrogrcun  may  have  had  on  those 
variables. 


A.    Definition  of  the  Mission  of  the  Organization 

Four  perspectives  on  the  question  of  defining  the 
mission  of  the  organization  will  be  made;  these  perspec- 
tives are  provided  by  three  items  from  the  content 
analysis  and  one  item  from  the  questionnaire. 

1.  Definition  of  the  Institution's  Mission 

Interview  Question:    VJhat  do  you  think  you  will 
obtain  (obtained)  from  the  AMA's  train- 
ing program? 

Range  of  Scale  Possibilities:     (1)  no  value 
to  (7)  maximum  value. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:    Experimental  State  Only  (El  and  E2) . 

2.  Sense  of  SED  Mission 

Inter %iew  Question:    What  are  some  of  the  road- 
blocks to  change  in  this  organization? 
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Range  of  Scale  Possibilities:     (1)  major  road- 
block/always stops  change  to  (7)  weak 
roadblock/seldom  stops  desired  change. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:    Experimental  States  (El  and  E2)  and 
Control  (CI). 

3.  Feelings  about  the  Direction  the  Organization 
is  MovingT" 

Interview  Question:    How  do  you  feel  about  the 
direction  your  organization  is  moving? 

Range  of  Scale  Possibilities:     (1)  not  satisfied 
at  all  to  (7)  completely  satisfied. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:    El,  E2,  CI. 

The  Questionnaire  item  is: 

4.  The  Kinds  of  Things  I  am  Doing  will  Make  a  Long- 
Term  Contribution  to  Education. 

Points  of  Time:    Tl,  T2,  T3,  T4. 


Ill 


Item     1  :    Definition  of  the  Institution's  Mission 
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Item  ^l,  "Dofinition  of  tho  Institution's  Mission/' 
was  applicable  only  to  tho  two  oxpGrimont^.l  groups.  In 
tho  Fall  of  1970  (when  data  was  gathered  prior  to  the 
training  program)  no  statistically  significant  differencos 
existed  between  the  two  Experimental  States  in  the  amount 
of  emphasis  given  to  defining  the  institution's  mission. 
Neither  did  any  differences  exist  in  the  Soring  of  1971, 
Pall  of  1971,  or  Spring  of  1972  (when  data  was  gathered 
after  the  program  had  been  executed).    Analysis  of  each 
state  ovar  time,  Fall,  1970  to  Spring,  1972,  revealed  that 
no  change  in  emphasis  occurred  in  Experimental  State  #1. 
A  statistically  significant  change  did  occur  in  Experi- 
mental State  #2;  this  change  was  due  to  decreased  emphasis 
being  placed  on  defining  the  institution's  mission  as  an 
effect  of  AMA  training.     Statistically  significant  change 
in  the  one  state  was  not,  however,  sufficient  to  produce 
differences  between  the  Experimental  States  in  the  post- 
training  periods. 

The  test  of  proportions  was  also  applied  to  this 
item  in  an  effort  to  annwer  the  question,  "Was  awareness 
of  this  variable  (independent  of  emphasis)  different 
between  the  states,  and  lid  it  change  over  time?"  Analysis 
determined  that  Experimental  State  #1  was  more  aware  than 
Experimental  State  #2  of  this  variable  prior  to  training 
(in  the  Fall  of  1970);  but  that  this  difference  was  not 
present  in  the  post-training  periods  (Spring,  1971,  Fall, 
1971,  and  Spring,  1972).    Neither  were  there  any  significant 
changes  in  awareness  when  the  two  States  were  analyzed 
over  time  (Fall,  1970  to  Spring,  1972). 

Therefore,  training  had  the  effect  of  decreasing 
the  amount  of  emphasis  placed  on  the  value  of  defining  the 
institution's  mission  in  Experimental  State  #2  but  had  no 
effect  in  Experimental  State  ftl.    t^Siile  there  were  some 
pre-training  differences  in  awareness  of  this  variable 
with  Experimental  State  #2  being  more  aware  than  Experi- 
mental State         neither  was  more  aware  than  the  other  in 
the  post-training  periods  nor  did  either  change  over  time. 
(In  this  context  "awareness"  simply  means  the  number  of 
people  who  provided  data  for  this  variable.) 

From  the  viewpoint  of  research  design  item  2,  "Sense 
of  SEP  (State  Education  Department)  Mission,"  provides  a 
more  valid  basis  for  evaluation;  data  for  this  item  was 
gathered  from  all  three  States. 

The  degree  of  emphasis  given  to  this  domain  as  a 
roadblock  to  change  was  not  affected  by  the  training  pro- 
gram.   Comparison  of  each  Experimental  State  with  the 
Control  State  both  before  and  after  training  revealed  no 
differences  between  any  of  the  States.    And  analysis  of 
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each  State  through  time  (Pall,  1970  to  Spring,  1972)  also 
showed  no  significant  change  concerning  the  degree  to 
which  the  presence  or  lack  of  a  clear  sense  of  SED  mission 
was  seen  as  a  roadblock  to  organizational  change. 

In  terms  of  awareness,  there  were  significant  dif- 
ferences between  the  States  before  training  (Fall,  1970). 
Experimental  State  #2  (hereinafter  called  E2)  weus  more 
aware  than  Experimental  State  #1  (hereinafter  called  El) 
of  the  sense  of  SED  mission  being  a  roadblock  to  change; 
and  both  Experimental  States  were  more  aware  than  the  Con- 
trol State  (hereinafter  called  CI) at  this  time.  After 
training,  these  differences  no  longer  existed. 

When  each  State's  awareness  was  assessed  over  time 
(Fall,  1970  to  Spring,  1972),  significant  changes  were 
observed  in  El  and  E2.    Both  States  experienced  decreases 
in  their  levels  of  awareness  of  this  variable.    There  was 
no  change  in  CI. 

The  amount  of  emphasis  placed  on  this  variable  as 
a  roadblock  to  change  was  not  affected  by  the  training;  in 
both  Experimental  States  training  merely  reduced  awareness 
of  this  variable. 

Item  #3,  "How  do  you  feel  about  the  direction  your 
organization  is  moving?"  was  asked  in  all  three  States. 
No  differences  in  satisfaction  with  organizational  direc- 
tion existed  between  El  and  CI  before  training.    In  one 
of  the  three  time  periods  after  training  (Fall,  1971),  El 
felt  significantly  better  about  the  direction  of  their 
organization  than  did  CI. 

Differences  did  exist  between  E2  and  CI  prior  to 
training;  this  difference  was  attributed  to  CI  which  re- 
flected greater  satisfaction  than  E2.    Immediately  after 
training  (Spring,  1971) ,  this  difference  was  even  greater 
but  then  lost  statistical  significance  in  the  last  two 
periods  (Fall,  1971  and  Spring,  1972). 

Comparison  of  El  and  E2  for  both  pre-  and  post- 
training  periods  indicated  greater  satisfaction  with  the 
direction  in  which  organization  El  was  moving  than  was 
the  case  in  E2.    These  differences  were  greater  immediately 
after  training  (Spring,  1971)  than  before  (Fall,  1970) 
although  differences  narrowed  as  time  passed  (Fall,  1971 
and  Spring,  1972) . 

Independent  analysis  of  each  of  the  three  States 
showed  no  statistically  significant  change  between  the 
Fall  of  1970  and  the  Spring  of  1972. 
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Since  all  interviewees  responded  to  this  question 
in  each  of  the  points  of  time  and  comparison,  no  test  of 
differences  in  awareness  could  be  made. 

Item  #4;     Fall    1970  to  Spring  1971 

Item  #4,  The  kinds  of  things  I  am  doing  will  make 
a  long-term  contribution  to  education,  a  questionnaire  item/ 
revealed  no  significant  differences  existed  between  either 
El  and  the  Control  State  or  E2  and  the  Control  State  which 
could  be  attributed  to  either  training  (represented  by 
column  F  on  the  previous  tables)  or  differences  between  any 
combination  of  these  States  (represented  by  row  F  on  the 
previous  tables) .    The  degree  to  which  people  in  the  States 
believed  they  were  doing  things  which  would  make  a  long- 
term  contributicm  to  education  did  not  change  when  the  pre- 
and  post-training  periods  are  compared  (column  F  in  the 
tables);  and,  the  relative  strength  of  belief  between  the 
States  (row  F  in  the  tables)  was  not  statistically  differ- 
ent in  either  Fall,  1970  or  Spring,  1971. 

Item  #4;     Spring,   1971  to  Fall  1971 

Neither  El  nor  E2  changed  their  opinions  of  their 
contribution  to  education  between  Spring,  1971  and  the  Fall 
of  1971.    They  felt  about  as  good  in  T2  as  they  did  in  T3. 
When  compared  to  CI,  the  Experimental  States  rovoaled 
no  significant  differences  either.    This  lack  of  any 
changes  over  time  or  differences  between  the  States  indi- 
cates that  training  had  no  effect  on  El  and  E2  at  this  time. 

Item  Us    Fall.   1971  to  Spring  1972 

Being  able  to  analyze  El  and  E2  in  contrast  to  a 
second  Control  State  (C2)  as  well  as  to  CI  gives  us  more 
confidence  that  we  are  getting  an  accurate  picture  of  what 
happened  in  the  Experimental  States  as  a  result  of  training. 
What  we  see  is  that  the  Experimental  States  did  not  change 
in  their  opinion  of  how  much  they  were  doing  for  education; 
however,  in  relation  to  CI,  El  indicated  more  pride  in 
their  work  while  E2  thought  they  had  done  less  than  C2. 
Nevertheless,  these  comparative  differences  had  not  changed 
between  Fall,  1971  and  Spring,  1972.    Thus  nothing  can  be 
credited  to  the  AMA  training  program. 

Item  #4:     Fall    1970  to  Spring,  1972 

This  is  the  key  test  of  program  effects*  Unless 
data  clearly  slioy  that  these  organizations  are  different 
after  than  they  vrere  before  training  began,  what  they  did 
at  Hamilton  cannot  be  said  to  have  made  any  lasting  impact 
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on  the  States.    Using  this  decision  rule,  it  is  obvious 
that  the  AMA  had  no  effect  on  El's  self-concept  of  the 
importemce  of  their  work  while  it  had  a  negative  influence 
on  £2.    Not  only  was  E2's  opinion  of  themselves  falling 
faster  than  Cl's,  but  E2  was  becoming  more  unstable  in 
the  extent  to  which  persons  in  E2  were  achieving  consensus 
on  this  point. 


DATA  SUMMARY 

Definition  of  the  Mission 
of  the  Organization 


Fall,  1970  -  spring,  1972  ^^^^  TRAINING 


Item 

Type  of  Data 
CONTENT 

Positive 
Effect 

No 
Effect 

Negative 
Effect 

1 

Definition  of  the  Institu- 
tion's Mission 

El 

E2 

2 

Sense  of  SED  Mission 

E1,E2 

3 

Feelings  about  the  direc- 
tion the  organization  is 
moving 

QUESTIONNAIRE 

E1,E2 

4 

The  kinds  of  things  I  am 
doing  will  medce  a  long- 
term  contribution  to 
education. 

El 

E2 

Training  had  no  effect  on  the  value  given  to 
defining  the  institution's  mission?  in  fact,  it  reduced 
it  in  E2.    Considered  as  a  roadblock  to  change,  the  train- 
ing again  had  no  effect  (positive  or  negative)  in  changing 
the  degree  to  which  this  was  a  problem.    The  tests  for 
awareness,  as  reflected  by  the  number  of  people  who  pro- 
vided data  for  this  research  category,  revealed  that  El 
and  E2  became  less  aware  of  this  variable  as  a  roadblock. 
This  does  not  mean  emphasis  changed  in  either  Experimental 
State;  it  only  means  that  significantly  fewer  people  men- 
tioned it.    Peelings  of  satisfaction  with  the  direction 
the  organization  was  moving  did  not  change  as  a  result  of 
training;  initial  differences  which  existed  prior  to  train- 
ing in  El  and  E2  were  eliminated  as  a  result  of  training. 
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The  questionnaire  item  which  focused  on  the  degree 
to  which  people  in  the  organizations  felt  they  were  making 
a  loncj-term  contribution  to  education  revealed  a  different 
pattern.    No  training  effects  were  observed  in  El  but 
persons  in  E2  felt  that  the  worth  of  their  contributions 
to  education  had  declined  xn  the  pre-  to  post-training 
period.    Before  training  began,  E2  believed  they  were 
making  a  greater  long-term  contribution  than  did  El  people; 
a  year  after  training  had  ended,  E2  felt  they  were  making 
a  lessor  contribution  than  El. 

Overall,  we  conclude  that  the  training  program  had 
no  effect  on  attitudes  about  how  much  a  sense  of  the  SEDs' 
mission  constituted  an  obstacle  to  organizational  change, 
or  hov;  satisfied  people  were  with  the  direction  of  their 
organization  in  El  or  E2.    Nor  did  training  affect  El's 
attitudes  about  defining  the  institution's  mission  or  about 
how  great  a  contribution  to  education  the  respondents  were 
making.    But  training  had  the  effect  of  downgrading  the 
value  of  defining  institutional  mission  and  their  own 
contributions  to  education  in  E2. 


B.    The  Development  of  Organizational  Objectives 
and  Priorities 

The  nine  items  below  specifically  address  the  P.m.' s 
training  goals  for  +-he  development  of  objectives  and  pri- 
orities.   The  first  four  items  are  taken  from  the  content 
analysis  and  the  next  five  from  the  questionnaire.    As  the 
first  three  items  are  content  categories  of  the  same  inter- 
view question,  comments  below  apply  to  all  three. 

1.  Modify  previously  established  objectives 

2.  Identify  and  analyze  alternative  courses  of  action 

3.  Determine  priorities 

Interview  Question:    What  do  you  think  you  will 
obtain  (have  obtained)  from  the  AMA's 
training  program? 

Range  of  Scale  Possibilicies:     (1)  no  value  to 
(7)  maximum  value. 

Points  of  Time:    Tl,  T2,  T3,  T4 , 


States:    El  and  E2  only. 
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4.  Do  you  feel  that  the  objectives  doveloped  as  a 
result  of  AMA  training  reflect  the  most  sirious 
and  pressing  needs  of  state  education? 

Interview  Question:    Same  -  Question  acts  as 
content  domain  in  this  case. 

Range  of  Scale  Possibilities:     (1)  not  at  all 
to  (7)  definitely. 

Point  of  Time:    T3,  T4. 

States:    El  and  E2  only. 

The  Questionnaire  items  eure: 

5.  The  goals  of  this  organization  are  articulated. 
Points  of  Time:    Tl,  T2,  T3,  T4. 

6.  Our  goals  are  realistic  and  attainable  with  our 
best  efforts. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

7.  The  top  priority  objectives  of  state  education 
are  clear  to  me. 

Points  of  Time:    T3,  T4. 

8.  I  feel  that  the  objectives  developed  during  Arm 
tralnlna  reflect  the  most  serious  and  pressing 
needs  of  state  education" 

Points  of  Time:    T3,  T4. 

States:  El  and  E2  only.  As  this  and  the  follow- 
ing item  refer  specifically  to  At!A  training 
they  were  not  asked  of  the  control  states. 

9.  As  I  see  it,  the  operational  priorities  of  the 
oFyictives  developed  during  AMA  training  are  clear. 

Points  of  Time:  T3,  T4. 
States:    El  and  E2  only. 
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Item    1    :  Modify  previously  established  objectives 


Fall.  1970 

Spring,  1971 

Fall.  1971 

Spring,  1972 

S  E2 

Ej      §  E2 

5  E2 

N 
4 

N 
8 

N 

11 

N 

11 

N  N 
7  8 

N 

7 

N 
9 

Kruska 

L-Wallis  One-Way  Analysis  of  Variance' 

H=  0.028 
Sie.  =  NS 

H=0.  475 
Sie.=  NS 

H=  0.656 
Sie.=  NS 

H=1.750 
Sig.  =  NS 

Binomial  Test  of  Proportions 

P=0. 0374 
SiK.=  .05 

P=0.7  02 
.    Sig.=  NS 

P=0.872 
Sig.=  NS 

P=0.450 
Sig.  =  NS 

Fall  1970         to               Soring  1972 

E,            §  E^ 

E2          §  E2 

N 
4 

N 

7 

N 

8 

N 
9 

Kruska 

1-Wallis  One-Way  Analysis  of  Variance 

H=1.508 
Sifl  =  NS 

H=  3.8912 
Si2.=  .05 

Binomial  Test 

of  Proportions 

P=  0.000966 
Sig.= . 01 

P=  0.017 
Sig.  =  .05 
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Item  2    :   Identify  and  analyze  alternative  courses  of  action 


Fall. 

1970 

Spring,  1971 

Fall, 

1971 

Spring, 

1972 

^2 

^2 

'  ^2 

^1  ' 

i  E2 

N 

N 

N 

N 

N 

N 

N 

N 

6 

6 

6 

8 

4 

4 

6 

6 

Kiiiska 

L-Wallis  One- Way  Analysis  of  Variance 

H=  1. 

256 

H=  1 

.350 

H=l. 

020 

H=  3 

.102 

Sie.= 

:NS 

1  Si2.= 

=  NS 

Sig.= 

:  NS 

Sie.= 

=NS 

Binomial  Test  of  Proportions 

P=0.774 

P=0. 

354 

P=0. 

962 

P=0 

.880 

SiK.= 

!  NS 

.  Sig.= 

s  NS 

Si«.= 

=  NS 

SiK.= 

=  NS 

Fall  1970         to               Sorinp  1972 

h            «  ^1 

E2           5  E^ 

N 
6 

N 
6 

N  N 
6  6 

Kruska. 

.-Wallis  One-Way  Analysis  of  Variance 

H=  1.442 

=  NS 

■H=  2.5641 

Si2.=  NS 

Binomial  Test  of  Proportions 

p=  0.089 
Sig.=  NS 

P=  0.171 
Sig.=  NS 
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Item  _3  :    Pete: nine  priorities 


Fall.  1970 

Spring,  1971 

Fall.  1971 

Spring.  1972 

§  E2 

§  E2 

Ej      §  E2 

Ej      §  E2 

N 
5 

N 
7 

N 

10 

N 
9 

N  11 
9  3 

N 
6 

N 

10 

Kruska 

L-Wallis  One-Wa] 

{  Analysis  of  Variance 

H=  0.949 
Si2.=  NS 

H=0.201 
Sie.=  NS 

H=  0.148 
Sie.=  NS 

H=  0.752 
Sig.=NS 

Binomial  Test  of  Proportions 


P=0.378 

P=0.644 

P=0.000 

P=0.002 

Sig.=  NS- 

.    Sig.  =  NS 

Sig.=  NS 

Sig.=  .001 

Fall  1970        to              Sorine  1972 

E,            §  E^ 

E2  6 

N 
5 

N 
6 

N 
7 

N 
10 

Kitiskaj 

L-Wallis  One-Way  Analysis  of  Variance 

H=  4.408 
qia  =  .05 

H=  5.485 
SiB.=-02 

Binomial  Test  of  Pi^portions 

P=  0.066 
Sig.=NS 

P=  0.000001 
Sig.  =  .001 

ERJC 
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Do  you  feel  that  the  objectives  developed  as 
a  result  of  AMA  training  reflect  the  most 
serious  and  pressing  needs  of  state  education? 


5  E2 


N 


Kruskal 


N 

10 


fipring,  1972 


§  E2 


N 
9 


Manic  fliiAJg^ 


H=  5.041 
Sig.=  .05 


N 

10 


Ana  1  yen  c  »f  Vay^ 


H=  2.535 

Sig,=  NS 


P=  1.000 

P=  1.000 

Sig.=  NS 

Sig.=  NS 

Fall. 

1971      to    Snrinp.  197 

^2 

Ej  § 

^2 

N 

N 

N 

N 

9 

9 

10 

10 

Kruskal-KaUisJOnedHajt 


H=  0.124 
Sig.»  NS 


H=  0.142 


Sig.=  NS 


Binomial  Twst  of  Prnpnrtinns 

p«  1.000     I  p=  i.ooo 


Sig.=  NS 


I*'  Prnport 
P=  l.OC 

I  Sig.=  NS 
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I tem        7         The  top  priority  objectives  of 

state  education  are  clear  to  me. 


Fall,  1971 
T 

Spring,  1972 

Experimental  SEDIl 

N 

68 

X 

5.617 

SD 
1.425 

N 

40 

5. 

X 

400 

SD 
1.296 

ExperiaMntal  SED#2 

45 

4:800 

1.575 

%0 

4. 

12^ 

1.552 

Control  SED  #1 

67 

5.417 

1.195 

61 

5. 

016 

1.431 

Control  SEr  «2 

61 

5.295 

1.406 

66 

4. 

969 

1.380 

Total 

241 

207 

TWO  Way  Analysis  of 
Variance 

T3I 

s  • 

F 

Simif. 

Experimental  SED  #1 

;oi. 

2.989 

NS 

•r/Control  SED  *1 

I^ow 

2.652 

NS 

Ejqperiaental  SED  #1 

Col. 

2.151 

NS 

w/Control  SED  «  2 

Row 

4.136 

05 

Experinental  SED  #2 

Col. 

1.437 

NS 

w/Control  SED  #1 

Row 

5.232 

.05 

Eiqperimental  SED  #2 

pol. 

0.949 

NS 

w/Control  SED  *2 

low 

3.241 

NS 
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Item  _L 


I  feel  that  the  objectives  developed  during 
AMA  training  reflect  the  most  serious  and 
pressing  needs  ot  state  education. 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  SED#1 

N 
68 

X 

4.926 

SD 
1.887 

N 

39 

X 

5.333 

SD 
1.131 

Experimental  SED#2 

4.333 

1.381 

39 

4.178 

1.555 

Total 

113 

78 

Two  Way  Analysis  of 
Variance 


T,  §  T, 


Experimental  SED#1 
W/Experinental  SED#2 


0.024 


Signif. 


iQQl 
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Item  _2. 


As  I  see  it,  the  operational  priorities  of 
the  objectives  developed  during  AMA  training 

eii^6  aisaif. 


Experimental  SED#1 
Experimental  SED#2 
Total 


Fall,  1971 
T3 


N 

68 


17 


11- 


X 

4.558 


4.ioo 


SD 
1.887 


Spring,  1972 
T4 


N 
37 


39 


76 


X 

5.108 


3.589 


SD 
1.264 


1.516 


Two  Way  Analysis  of 
Variance 


T,  §  T, 


Experimental  SED#1 
W/Experimental  SED#2 


lol 


0.015 


15.065 


Sienif . 


.QQl 
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There  were  no  significant  differences  between  the 
two  Experimental  States  in  the  degree  of  emphasis  given 
to  modification  of  previously  established  objectives  as  a 
possible  benefit  of  AMA  training.    Prior  to  training  E2 
was  more  aware  than  El  than  modification  might  be  a  result 
of  training^  but  this  difference  disappeared  immediately 
after  training  and  for  the  two  subsequent  periods  of  this 
evaluation.    The  changes  in  awareness  observed  within  El 
and  E2  between  the  Fall  of  1970  and  the  Spring  of  1972  are 
not  what  they  seem;  the  recorded  significance  levels  are 
due  to  the  greater  number  of  respondents  in  T4^  not  to 
increasing  awareness  of  this  variable.     However^  negative 
training  effects  did  occur  in  E2  between  Tl  and  T4;  this 
organization  indicated  that  their  objectives  had  been 
modified  after  training  much  less  than  they  had  expected 
the  objectives  to  be  modified  before  training.    The  AMA 
program  was  held  accountable  for  this  failure  to  meet  E2 
expectations . 

During  the  course  of  this  evaluation  no  significant 
change  took  place  with  regard  to  item  #2.     The  top  managers 
of  the  Experimental  States  experienced  no  impact  from 
training  upon  their  ability  to  identify  and  analyze  alter- 
native  courses  of  action. 

Item  #3  concerning  the  determination  of  priorities 
shows  no  significant  difference  between  the  Experimental 
States  during  the  period  Fall  1970  to  Spring  1972.  Within 
both  states  there  was  a  significant  change:     by  Spring  19  72 
the  top  managers  in  both  states  placed  less  emphasis  on 
having  developed  priorities  due  to  AMA  training  than  they 
had  before  training.     There  was  a  significant  increase  in 
the  awareness  of  priorities  in  E2,  but  at  the  same  time  the 
managers  emphasized  this  item  less  and  less  as  time  passed. 

Item  #4  was  asked  of  the  top  managers  in  Fall  1971 
and  Spring  1972  only.     The  objectives  developed  in  El  were 
seen  as  more  relevant  by  those  managers  than  the  goals 
developed  in  E2  in  Fall  1971.    This  difference  between  the 
states  no  longer  existed  in  Spring  1972.    Managers  in  both 
states  viewed  the  objectives  developed  as  having  the  same 
degree  of  importance.    No  difference  existed  within  the 
states  over  the  value  of  objectives  developed  during  this 
period. 

Item  #6:     Our  goals  are  realistic  and  attainable  with  our 
best  efforts. 

Fall  1970  to  spring  1971 

On  this  item  State  El  in  comparison  to  Control  1 
showed  no  effects  of  training.     Holding  time  constant^ 
the  two  states  differed  from  each  other.     State  E2  did 
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experience  a  significant  training  effect.    TJhile  no 
significant  differences  appear  between  E2  and  Control  1, 
significant  change  over  time  did  occur.    Both  states  felt 
their  goals  were  less  realistic  over  time,  but  E2  experi- 
enced more  st2Jl)ility  than  Control  1,  which  experienced 
increasing  instability. 

Snrin?  1971  to  Fall  1971 

During  this  period  State  El  experienced  a  negative 
training  effect.    Vlhen  compared  to  the  Control  State  1 
El  experienced  a  slower  increase  in  the  belief  that  its 
goals  were  realistic  and  greater  instability. 

State  E2  also  experienced  a  negative  training  effect. 
Significant  change  occurred  over  time  and  within  El  and 
Control  1  with  both  states  experiencing  increasing  mean 
scores  at  about  the  same  rate.    However,  in  relation  to  the 
Control,  E2  was  growing  stable  at  a  much  slov/cr  rate. 

Fall  1^71  to  Spring  .972 

No  significant  differences  occurred  between  El  and 
either  Control  1  or  Control  2  during  this  period.  Signifi- 
cant change  did  occur  between  E2  and  Control  1.  Both 
states  experienced  weciker  support  of  their  organizational 
goals,  but  E2  experienced  significantly  less  support  than  CI 
together  with  greater  instability.    This  indicates  a  nega- 
tive training  effect. 

F<?J1  1^70  to  Spring  1972 

State  El  experienced  no  training  effect  concerning 
this  item.    That  is,  the  respondents  in  this  organization 
sav;  their  organization's  goals  as  no  more  realistic  or 
attainable  in  the  Spring  of  1972  than  they  had  prior  to 
training. 

State  E2  also  did  not  experience  any  effect  of 
training  on  this  item.    The  significant  difference  between 
State  E2  and  CI  continued  but  did  not  change  over  time 
significantly,  indicating  no  training  effect. 


Item  7:    The  top  priority  objectives  of  State  Education 
are  clear  to  me> 

Fall  1$71  to  Spring  1972 

This  item  was  asked  in  Y2  only.    Over  this  period 
the  clarity  of  priorities  decreased  in  all  states,  though 
this  chamga  was  insignificant.    No  differences  existed 
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between  State  El  and  Control  1  but  did  exist  between  State 
El  and  Control  2.    State  E2  and  Control  1  also  shovred  a 
significant  difference  concerning  the  amrhasis  given  this 
item.    No  difference  existed  between  State  E2  and  Control  2. 


Item  8:  I  feel  that  the  objectives  doveloped  during  M'A 
training  reflect  the  most  serious  and  pressing' 
needs  of  3tate  Education 

Fall  1971  to  Spring  1972 

This  item  was  asked  in  the  Experimental  States  during 
y2.    Mo  significant  changes  over  time  were  found  within 
the  states  regarding  the  value  of  the  objectives  developed 
but  there  were  significant  differences  between  the  states. 
State  El  saw  the  objectives  as  being  more  relevant  than  did 
State  E2.    This  difference  became  greater  in  the  interval 
between  the  two  periods  of  questioning.    State  El  liked 
the  objectives  developed  more  in  the  Spring  of  1972  than 
they  had  in  Fall  1971?  State  E2  liked  them  less  in  the 
Spring  than  they  had  during  the  previous  Fall. 


Item  9:    As  I  see  it,  the  operational  priorities  of  the 

objectives  developed  during         training  are~clear. 

Fall  1D71  to  Spring  1972 
This  item  follows  the  same  pattern  as  item  8  above. 
Again,  while  no  statistically  significant  differences  over 
time  existed  within  the  states,  there  were  significant 
differences  between  them.    State  El  again  saw  the  obaectives 
more  favorably  than  did  State  E2  and  this  difference 
increased  with  time.    State  El's  reaction  to  this  item  was 
growing  more  positive  with  time  while  State  E2's  was  beccan- 
ing  more  negative.    In  other  words,  the  staff  in  State  E2 
were  fliore'  clearly  c6mpr*h*4nding  the  operational  priorities 
demanded  by  the  objectives,  while  the  personnel  of  State 
E2  were  becoming  less  clear. 


Item  5:    The  goals  of  this  organization  are  articulated. 

Fall  1?10  to  Spring  1971 

This  item  shows  no  effect  of  training  in  State  El 
when  compared  with  the  Control  although  the  States  were 
significantly  different  from  each  other  if  time  is  held 
constant.    E2  when  compared  with  the  Control  shows  differ 
ences  as  a  result  of  training  ?jid  between  the  States. 
Goals  in  E2  were  significantly  less  clear  after  training 
than  thoy  had  been  before  it. 
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Goring  1971  to  Fall  1971 

The  degree  to  which  articulated  goals  were  thought 
to  exist  increased  in  all  the  states  between  the  Spring 
and  Fall  of  1971.    State  El  experienced  a  positive  effect 
of  training;  it  significantly  increased  its  goal  clarity 
in  relation  to  the  Control  State.     State  E2  also  showed  a 
significant  effect  of  training  with  its  goals  being  more 
articulated  over  time  although  it  was  not  significantly 
different  from  the  Control,  holding  time  constant. 

Fall  1971  to  Spring  1972 

The  clarity  of  the  goals  in  all  the  organizations 
decreased  over  this  period  of  time.    No  significant  train- 
ing effects  were  noted.    There  were  significant  differences 
between  State  Fl  and  both  Control  States  holding  time 
constant.    El's  goals  vrore  more  articulated  than  either  of 
the' Control  States,  but  this  did  not  change  significantly 
over  time  b-^tween  El  and  either  of  the  Control  States. 

State  E2  and  Control  State  1  experianced  a  change 
over  time.    Significant  differences  over  time  did  occur 
but  v'ith  no  significant  change  between  the  States.  State 
E2  and  Control  1  both  were  lass  clear  regarding  their  goals, 
with  E2  decreasing  at  a  faster  rate.    But  CI  experienced 
more  instability  than  E2.    No  differences  existed  between 
E2  and  Control  2. 

Fall  1570  to  Spring  1972 

A  comparison  of  the  pre-training  period  with  the 
last  intervention  shovred  no  significant  training  effects 
in  either  Experimental  State  regarding  the  degree  to  which 
goals  were  articulated  within  the  organizations.    State  El 
and  Control  1  showed  no  significant  differences  over  time 
or  in  the  degree  of  goal  articulation  experienced  by  each 
state . 

State  E2  and  Control  1  showed  no  change  over  time 
concerning  the  cl^gree  to  which  these  states  saw  their  goals 
as  being  articulated.    The  states  significantly  differed 
frc:.\  each  other  at  both  points  in  time  concerning  the 
emphasis  given  this  item. 
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DATA  S0I4MARY 
Development  of 
Organizational  Objectives 
and  Priorities 

Fall,  1970  -  Spring,  1972 


Item 


Type  of  Data 
CONTENT 


IMPACT  OP  TRAINING 


Positive 
Effect 


No 
Effect 


Negative 
Effect 


1 

Modify  previously  estab- 
lished objectives 

.El 

E2 

2 

Identify  and  analyze 
alternative  courses  of 
action 

E1,E2 

3 

Determine  priorities 

E1,E2 

4 

Do  you  feel  that  the 
objectives  developed  as 
a  result  of  AT^A  training 
reflect  the  most  serious 
and  pressing  needs  of 
state  education? 

QUESTIONNAIRE 

• 

E1,E2 

5 

The  goals  of  this  organi- 
zation are  articulated. 

E1,E2 

6 

Our  goals  are  realistic 
and  attainable  with  our 
best  efforts. 

E1,E2 

7 

The  top  priority  objec- 
tives 0,^  state  education, 
are  clear  to  me. 

E1,E2 

8 

I  feel  that  the  objectives 
developed  during  AMA  train- 
ing reflect  the  most  seri- 
ous and  pressing  needs  of 
state  education. 

E1,E2 

9 

As  I  see  it,  the  operation- 
al priorities  of  the  ob- 
jectives developed  during 
AMA  trairing  are  clear. 

E1,E2 
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/inalysis  of  these  nine  items  indicates  that  overall 
7VMA  training  exerted  negligible  effect  on  how  organization 
members  felt  about  their  objectives  and  priorities* 

The  two  exceptions — organizations  v/here  training 
effects  were  found — involved  content  categories ^  and  both 
cases  were  negative.    The  program  did  not  modify  E2 
objectives  (Item  1)  nearly  so  visibly  as  persons  within 
that  organization  had  predicted  before  training.  This 
gap  between  promise  and  performance  constituted  a  negative 
training  effect.    The  other  exception  refers  to  the  deter- 
mination of  priorities  (Item  3);  both  Experimental  States 
placed  greater  value  on  this  variable  prior  to  training 
than  was  given  after  training. 

*    These  findings,  however,  should  be  interpreted  with 
all  of  the  caution  appropriate  to  a  research  design  which 
does  not  include  at  least  one  control  group. 

None  of  the  questionnaire  items  revealed  any  influ- 
ence of  the  AIIA  progr^.     It  is  interesting  to  note  that 
El  thought  more  of  their  objectives  and  priorities  than 
E2  during  ovcry  time  period  covered  by  these  items;  however , 
these  differences  wore  relatively  stable,  indicating  that 
they  were  not  attributable  to  training.    Those  items  that 
refer  specifically  to  the  AJIA  share  with  content  data  the 
interpretive  problem  inherent  in  the  lack  of  a  control  group. 

Despite  this  qualification,  the  research  team  is 
reasonably  certain  that  the  objectives  and  priorities  of 
the  SED's  which  participated  in  the  team  planning  process 
were  largely  unaffected  by  it. 


AREA  II:     CAUSAL  VARIABLES  8  THROUGH  13;  SED'S 

In  this  section,  wo  examine  five  training  goals 
(8-13)  the  ATIA  established  for  its  program,  the  extent  to 
which  they  have  been  accomplished,  and  the  attitudes  of 
the  participants  toward  the  goals.     In  the  first  part  of 
the  discussion  below,  we  examine  expected  results,  actual 
results,  and  the  m.?.jor  issues  raised  by  the  implementation 
of  criteria  8-13.    The  second  part  includes  a  data  display 
and  interpretation  of  expressed  attitudes  relevant  to  these 
criteria. 

Section  1;  Action 

A.    Expected  Results 

To  better  understand  what  AMA  training  accomplished 
for  the  organizational  planning  process,  we  should  first 
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review  AMA  evaluation  criteria  8-13.       Review  in  this  case 
means  listing  and,  where  further  explanations  are  neces- 
sary, defining  terms.    The  basic  Arv>  definitions  for  these 
terms  have  been  used. 

8.  Made  strategic  action  assignments; 

Action  Assignments:    Authoritative  allocation 
of  responsibility  "to  staff  member (s)  for 
specific  elements  of  the  plan,  planning  pro- 
cess, or  general  administration  of  the 
orgemization.    This  responsibility  becomes 
part  of  his  job  description  or  overall  defini- 
tion of  the  responsibilities  of  his  position. 

9.  Defined  standards  of  performance  for  key 
administrators ; 

Performance  Standards;    Statement  of  what 
will  result  if  stafr" member (s)  properly  dis- 
charge an  action  assignment. 

10.  Specified  task  completion  dates; 

Task  Completion  Pates;    Statement  of  when  an 
aefeton  assignment  is  expected  to  be  completed. 

11.  Designed  supplementary  planning  efforts? 

Supplementary  Planning  Efforts;  Supportive 
activities  undertaken  to  promote  the  imple- 
mentation of  the  planning  process  and  the 
plans  themselves.    These  can  include;  in- 
sarvice  training,  planning  guidelines,  com- 
munication of  planning  concepts  throughout 
the  agency,  etc. 

12.  Assigned  responsibilities  to  subordinate  units? 

Responsibilities  to  Subordinate  Units; 
Authoritative  allocation  of  responsioility 
to  an  organizational  unit  for  specific  parts 
of  the  plan  or  planning  process - 

13.  Designed  a  methodology  by  which  future  performance 
may   be    evaluated  in  relation  to  the  performance 
specified  in  the  plan? 

Evaluation  Methodology;    Process  by  which 
expected  can  be  compared  with  actual  results. 


'ama,  0£.  cit.,  pp.  4-5 
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B»    J^ctual  Results 

Here,  the  question  is^  what  has  been  done  to  satisfy 
these  criteria?    As  with  the  first  seven  goals,  we  are  not 
in  a  position  to  assess  the  relevance  or  accuracy  of  these 
last  six  where  actual  behavior  is  at  issue.    Our  concern  is 
whether  written  documents  were  developed,  and  what  form 
they  tock.    Again,  we  have  summarized  our  findings  6n  the 
following  chart ,  followed  by  an  explanation  of  each  item. 


No> 
8. 

9. 

10. 
11. 
12. 

13. 


SUMMARY  OP  ACTION  FINDINGS 
SED  LEVEL 
TiJlA  CRITERIA  8-13 
FAT.T.^  1970  to  SPRING,  1972 


Amount  of  Progress 


Criteria 


Minimum     Moderate  Maximum 


Made  strategic  action 
assignments  El 

Defined  standards  of 
perfoxrmance  for  key 
administrators  El 

Specified  task  comple- 
tion dates  El 

Designed  supplementary 
planning  efforts 

Assigned  responsibili- 
ties to  subordinate 
units 

Designed  a  methodology 
by  which  future  perfor-* 
mancG  may  be  evaluated 
in  relation  to  the  per- 
formances specified  in 
the  plan  El,E2 


E2 


E2 


E2 


E1,E2 


El,E2 


Strategic  action  assignments  (#8)  and  those  criteria 
closely  connected  with  it  (performance  standards  (#9)  and 
task  completion  dates  (#10))  can  be  considered  together. 
Deciding  who  should  do  what  and  when  are  interrelated 
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elements  o£  the  planning  process.    Those  decisions  arc- 
continuous  and  recur  throughout  the  planning  cycle;  for 
the  purpose  of  evaluation  this  process  can  be  divided  into 
two  phases. 

The  first  phase  occurred  during  and  immediately 
after  training;  it  involves  initial  conceptualizations  of 
the  planning  process  and  their  introduction  into  the  organi- 
zation.   The  second  phase  is  longer  term  and  assesses 
responsibilities  for  specific  parts  of  the  plans  subsequently 
developed. ^0 

Not  much  can  be  said  about  the  first  phase  except 
that  it  was  largely  accomplished.    At  this  early  stage, 
efforts  were  tentative,  preliminary  and  not  susceptible  to 
rigorous  judgment.    Frequently,  accomplishment  at  this  phase 
meant  the  gathering  of  inter session  information  or  explana- 
tion of  planning  concepts  prior  to  the  development  of 
departmental  plans. 

The  second  phase  is  intended  to  extend  over  a  longer 
Pc3riod;  it  is  to  serve  as  the  basis  for  quasi-permanent 
apportionment  of  powers  and  duties  within  the  orgeinization. 
After  plans  have  been  written,  parts  of  them  are  given  to 
the  appropriate  division.    Reading  progress  might  be 
assigned  to  instruction;  music  is  referred  to  cultural  arts, 
etc.    Each  division  in  turn  gives  parts  of  its  specific 
program  to  staff  within  the  division,  "sub-parts"  which 
refer  directly  to  AMA  criteria  8*10. 

Taken  together,  we  can  refer  to  them  as  "job 
descriptions."    Staff  agree  to  adhere  to  formulations  about 
the  position,  its  responsibilities,  an4  what  should  be 
accomplished  by  a  specific  date.    Action  assignments, 
performance  standards,  and  task  completion  dates  fom  a 
unit. 

Actual  second  phase  accomplishment  varies  with  the 
states.  EfSED  #2  has  heavily  emphasized  the  production  of 
job  descriptions;  several  divisions  have  them  on  paper. 
ESED  #1  has  made  little  progress  and  still  relies  on  "Request 
for  Position  Classification"  forms  which  have  been  used  for 
many  years. 21    This  by  no  means  implies  that  one  state  has 
done  better  than  the  other  in  respect  to  job  descriptions; 


An  example  of  a  first  phase  assignment  might  be 
to  gather  feedback  from  agency  personnel  on  a  reading 
objective.    If  the  objective  is  approved,  implemented  and 
given  to  a  particular  unit  or  individual  for  attainment, 
it  is  characteristic  of  the  second  phase. 

^^Porm  PD-118  (Revised  8-62). 
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it  does  reflect  different  approaches  to  the  dcvelopraent 
of  organizational  planning  however.    In  both  states  internal 
priorities  v/ero  established  regarding  the  development  of  a 
planning  system.    ESED  #1  concentrated  on  the  development 
of  a  Plan  for  Planning  while  ESED  #2  developed  job  descrip- 
tions.   The  results  of  following  these  different  approaches 
may  have  implications  for  the  implementation  of  planning 
which  we  discuss  in  the  Emerging  Issues  section  of  this 
chapter. 

In  any  event/  since  our  assignment  is  to  evaluate 
what  has  been  accomplished  we  intend  to  examine  documents 
written  in  both  states  pertinent  to  criteria  8-13.  For 
this  purpose  we  have  adapted  several  AMA  "Rules  for  Develop- 
ing Stand5rd£"22  and  applied  them  to  selected  job  descrip- 
tions.   Thcs'n  criteria  are: 


1.  CLfJlITY: 


2 .     f  lEASURABLE : 


3.  STRUCTURED: 


Clear  statements  of: 

1.1  action  assignments 
(responsibilities  given) 

1.2  performance  standards 
(results  expected) 

2.1  expressed  in  measurable  terms 

2.2  task  completion  dates 

Set  up  in  approved  form^ 

3.1  position  title  and  organi- 
zational unit 

3.2  purpose  of  position/general 
responsibilities 

3.3  assignments  directly  com- 
pared v;ith  performance 
standards 


Both  states  submitted  samples  of  what  they  charac 
terized  as  their  latest  job  descriptions.     ESED  #2's 
covered  most  of  thdir  executive  staff;  ESED  #1  scattered 
its  exhibits  across  several  organizational  levels. 


"^^James  L.  Hayes,  "Selecting,  Appraising,  and 
Developi-ig  Top  Management  Personnel,"  in  Top  Management 
Briefing  Manual,  The  President's  Association,  The  American 
Management  Association. 
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ASSESSMENT  OF  SELECTED  JOB  DESCRIPTIONS 


CRITERIA 


ESED  #1 


ESED  #2 


(1)  CLARITY 

1«1  Action 
Assignments 


1.2  Perfor- 
mance 
Standards 


Pound  on  2  points  on 
stemdardized  £orm: 
under  ''Explanation" 
(of  position)  and 
••Description  of  Work/ 
which  is  redundcint« 
Reasonably  clear  and 
understandable « 

Performamce  standards 
are  not  explicit. 


Most  appear  to  be  con- 
sise  and  understandable 
statements  of  position 
responsibilities;  con- 
sist of  short  sentences 
or  sentence  fragments 
in  series;  presented 
as  a  unit« 

Some  do  not  contain 
performance  standards; 
but  most  do«  Expressed 
in  terms  of  tasks  to  be 
ccxnpleted  rather  than 
ultimate  impact  on 
students* 


(2)  IlEASURABLE 

2«1  Measurcible  Action  assignments 
Terms  expressed  in  general 

terms ;  per f ormemce 
standards  not 
explicit* 


2.2  Task 

Completion 

Dates 


None 


Good  overall  with  some' 
overgeneralized*  Many 
performance  standards 
contain  %  attainment 
figures « 

Inserted  where 
appropriate* 


(3)  STRUCTURED 


ERLC 


3*1  Position 
Title  and 
Organxza- 
tional  Unit 

3«2  Purpose 
of  Position/ 
General 
Responsibili* 
ties 

3*3  Assign- 
ments dir- 
ectly com- 
pared with 
Performance 
Standards 


Present 


Present 


Present;  under 
"Explanation"  and 
"Description  of 
Work.** 


No  performance 
standards  with  which 
to  compare  action 
assignments « 


Present;  under  those 
tit lee « 


Directly  compared; 
assignments  are  on  one 
side  of  the  page;  per- 
formance standards 
are  on  the  other* 
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The  supplementary  planning  efforts  goal  is  a  catch-all, 
as  supportive  activities  often  are.    The  activities  needed  to 
introduce  and  implement  a  planning  system  are  myriad  and 
not  easily  categorized.    Everything  could  be  busywork  or  it 
could  be  vital  to  the  survival  of  the  system.    One  can  never 
be  certain  of  the  importance  of  supplementary  planning  until 
the  end  result  variables  are  analyzed  and  evaluated.  Sup- 
plementary planning  efforts  are  input  oriented;  final  assess- 
ments must  be  postponed  until  out:put  is  examined. 

But  supplementary  planning  efforts  are  essential  to 
fuel  the  planning  system  cind  keep  it  going.    Using  organi- 
zational reports,  we  can  list  some  activities  begun  to 
supplement  planning  since  the  training  ended. What  has 
been  done  is  considerable f  in  both  states.    Whether  these 
activities  are  '•good"  or  ••bad"  is  a  question  which  cannot  be 
amswered  within  the  confines  of  this  evaluation.  Indirect 
evidence,  however,  will  be  provided  when  we  look  at  what 
overall  planning  has  accomplished  with  the  assistance  of 
these  supplementary  efforts. 


Inventory  of  Sut>plementary  Planning  Efforts 

ESED  #1 

Educational  Planning  Studies 

1.  studena  unrest 

2.  legislative  priorities 

3.  educational  change  survey 

4^    barriers  to  educational  change 

5.  attitudes  of  students  toward  those  of  an  opposite  race 

6.  53  experimental  programs  to  improve  classroom  instruct 
tion  and  school  administration 

7.  8  experimental  programs  in  career  education 

8.  statewide  needs  assessment 

In-Service  Training 

1.  14  orientation  and  training  programs  within  the  SED 
including  a  three-day  planning  conference  for  all 
professional  personnel  in  program  services  to  develop  spe- 
cific objectives  for  the  1971-72  school  year. 

2.  orientation  and  training  sessions  for  LEA's  including: 


The  major  documents  utilized  to  survey  supplementary 
planning  efforts  included  Open-Ended  Questionnaires  completed 
by  the  chief  planning  officers  in  each  state.    In  addition 
memoranda  concerning  AMA  planning  weuce  «xiwladd|  In  particular: 
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A,  1971  Slimmer  Conference  for  all  local  superinten- 
dents and  assistant  superintendents  in  the  state 
(approximately  300)  to  develop  management  concepts 
and  practices, 

B,  15  superintendents'  districts  have  been  allocated 
staff  development  funds  cuid  are  conducting  manage- 
ment seminars, 

C,  3  projects  for  25  LEA*s  in  management  and  leader- 
ship development  have  been  funded  by  the  state 
board  of  education  with  funds  appropriated  by  the 
General  Assembly, 

D,  Title  III  ESEA  grant  is  being  implemented,  provid- 
ing opportunities  for  developing  management  and 
leadership  skills  in  8  school  systems. 

E,  Award  of  a  USOE  grant  to  train  6  trainers  who  in 
turn  will  help  develop  planning  competence  and 
plans  in  all  152  LEA's/ 

Planning  Guidelines 

1.  Plan  for  Planning^^ 

2.  HandbooK  for  Planninq;^^ 

Miscellaneous 

1.  leadership  school  for  secondary  school  pupils 

2.  drafting  of  plan  for  accountability 


ESED  #2 


Educational  Planning  Studies 

1.  early  childhood  education 

2.  reading  for  12  year  olds 

3.  human  relationships  in  the  schools 


ESED  #2    State  Superintendent,  "Memo  to  Professional  Person- 
nel:   Strengthening  the  Department"  (3  Nov.  1971);  and 
ESED  #1    Assistant  Superintendent,  "Memo  on  the  Status  of 
SEA  &  LEA  Participation  with  American  Management  Association" 
(14  Oct.  1971). 


2^! 
25, 


* State  Education  Agency,  A  Plan  for  Planning  (March, 

1971). 


ERiC 


^State  Education  Agency,  A  Handbook  for  Planning: 
Elementary  and  Secondary  Education"  tJuly^  1971). 


143 


4.  statewide  noods  assessment  (in  progress) 

5.  work  on  management  information  system  (MIS)  plan  and 
approach 

6*    state  evaluation  strategy 

7.    task  forces  working  on  specific  planning  problems 
In-Service  Training 

1.  management  and  planning  seminar  for  the  2  experimental 

lea's 

2.  one  half--day  orientation  session  for  the  Bureaus  of 
Educational  Programs  and  Administrative  Services 

3.  two-day  programs  for  aach  bureau  on  management  practices 
with  specific  emphasis  on  position  descriptions  and 
performance  standards 

Planning  Guidelines 

None  recent 

Miscellaneous 

1.  management  by  objectives  (MBO)  regulation  v/ith  depart- 
ment wide  planning  procedures  is  under  review 

2.  planning  and  budgeting  schedules  and  procedures  have 
been  established  for  developing  budget.    FY74  budget 
will  not  have  substantial  amount  of  new  information  based 
on  AMA  planning  format  as  this  crosswalk  has  been  post- 
poned. 

3.  drafting  and  passage  of  accountability  legislation. 27 

To  some  extent^  responsibilities  have  been  assigned 
to  subordinate  units  in  the  two  experimental  states.  Both 
have  re- emphasized  the  role  of  their  planning  units  by  add- 
ing  additional  staff  and  providing  more  money. 

Some  reorganization  has  also  occurred.    ESED  #2  has 
developed  an  executive  council  composed  of  the  superintendent, 
deputy  superintendent,  the  two  associate  superintendents^ 
and  the  coordinator  of  planning  (as  an  observer/adviser). 
This  group  oversees  the  entire  agency  on  a  consultative 
basis.    The  agency  has  been  organized  into  two  bureaus: 
Educational  Programs  and  Administrative  Services/  with  its 
own  council. 


In  1970 f  a  Planning  Council  within  ESED  #2  came 
up  with  "A  Design  for  Comprehensive  Planning"  (13  March  1970) 
but  it  was  never  really  implemented  within  the  agency.  The 
reason  given  for  this  lack  of  attention  was  that  it  was  too 
advanced  for  its  time  and  was  beyond  the  planning  competencies 
of  most  of  the  members  of  the  department  at  that  time. 

27 

Public  School  Laws  of  (  ),  Article  77,  Section  28A, 
"Educational  Accountability." 
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ESED  #1  has  reorganized  itself  into  six  broad  areas 
of  responsibility  with  an  assistant  suparintendent  over 
each.    These  persons,  plus  the  state  superintendent,  form 
the  executive  staff  of  the  department.    At  least  on  a  formal, 
stjnictural  basis,  both  states  have  stressed  the  management 
team  as  the  locus  of  decisionmaking. 

The  evaluation  system  is  probably  the  weakest  planning 
link    in  both  states.    Without  a  completed  needs  assessment, 
there  is  little  baseline  data  on  which  to  compare  educational 
programs  over  time.    This  will  become  clearer  in  the  atti- 
tudinal  data  displays,  wherein  organizational  members  were 
asked  to  state  what  specific  methods  they  used  to  measure 
whether  their  continuing  and  specific  objectives  were  being 
achieved.    While  work  is  going  forward  in  both  states,  ESED 
#2  probeibly  has  worked  more  assiduously  to  "design  a 
methodology  by  which  future  performance  may  be  evaluated  in 
relation  to  performance  specified  in  the  plan. "28 


C.    Emerging  Issues 

Many  of  the  issues  discussed  in  reference  to  criteria 
1-7  also  apply  here.    But  there  are  additional  questiom; 
which  have  special  relevance  to  this  section.    7is  in  tho 
preceding  section,  note  of  these  comments  and  criticisms 
does  not  necessarily  mean  that  the  research  team  agrees  with 
them.    We  are  reporting  what  seem    to  us  significant  issues 
that  were  raised  about  these  criteria. 

1.    Job  Descriptions  &  Plans;    A  Question  of  Timing 

We  have  already  indicated  that  one  of  the  reasons 
for  the  difference  in  ^SED  #1  and  #2's  comparative  work  on 
job  descriptions  was  that  ESED  #2  more  heavily  emphasized 
defining  positions  and  delineating  responsibilities  than  on 
finalizing  their  plans.    The  reverse  is  true  in  ESED  #1; 
plans  were  made  first  and  they  are  only  now  detailing  agency- 
wide  action  assignments  and  performance  standards. 

We  just  haven't  gotten  around  to  it. 
We  put  all  the  planning  ducks  in  a 
row  and  decided  to  take  a  pot  shot  at 
the  plans  first  and  save  performance 
standards  until  later.    I  guess  we 
could  have  nailed  dov/n  standards  but 


28 

See  for  example:    "Report  of  Federal  Financial 
Assistance  and  Application  for  Continued  Funding  for  the 
Planning  and  Evaluation  of  Educational  Programs"  (Section  402, 
General  Education  Provisions  Act,  Section  A-3,  Attachment  #1)." 

ERIC 
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WG  figured  it  would  make  more  sense 
.  .  •  to  decide  on  what  we're  going 
to  do  before  deciding  who's  going  to 
do  it. 29 


ESED  #2  has  spent  much  time  in  conferences^  task  forces ^ 
and  other  meetings  going  over  job  descriptions  for  executive 
and  professional  staff.    Their  feeling  was  that  it  would  be 
more  logical  to  carefully  define  what  everyone  is  now 
responsible  for  before  going  on  to  decide  v/hat  they  will  be 
responsible  for  under  a  new  plan.    Tightening  up  divisional 
lines  would  define  the  boundaries  within  which  personnel 
could  plan. 

Some  personnel  think  this  is  a  worthwhile  approach. 
Others  are  not  so  sure.    They  argue  that  the  sessions 
devoted  to  job  description  writing  were  a  waste  of  time, 
straitjacketed  all  participants,  and,  given  the  lack  of  a 
related  plan,  were  artificial. 

2.    Guidelines  for  Planning 

An  important  part  of  the  TMIi  planning  process  is  the 
development  of  guidelines  for  planning,  i.e.,  a  "blueprint" 
which  teaches  departmental  personnel  how  to  construct  a  plan. 

ESED  #1  went  to  work  immediately  after  training  and 
developed  standardized  planning  procedures  and  definitions 
for  use  throughout  the  department.    Many  persons  consider 
them  useful  and  they  are  a  clear  statement  of  how  to  go 
about  agency  planning. 

ESED  #2  postponed  formal  attention  to  planning 
guidelines  and  is  now  getting  around  to  setting  up  an 
"informal  task  force  to  answer  two  questions  which  the 
executive  council  posed:     (1)  What  is  the  department's  plan? 
and  (2)  How  do  you  get  it?    We've  come  up  with  two  or  three 
alternative  models,  and  a  list  of  standard  questions  that 
have  to  be  answered,  a  planning  road  map  if  you  will."f^ 
Planning  personnel  do  not  see  this  delay  as  a  shortcoming; 
instead  they  see  it  simply  as  "the  next  step  in  a  continuing 
evaluation  of  the  planning  process  for  the  department. "^-^ 

Some  managers,  however,  feel  that  they  need  some 
written  direction,  especially  the  managers    whose  AMA  train- 
ing was  minimal. 


^^Executive  Staff,  ESED  #1.  i 
^°ExecutivG  Staff,  ESED  #2.  | 

ERIC  ^^Ibid.  % 
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I'd  liko  to  SGC  us  get  something  that 
is  neat,  clean,  briof  that  everybody 
has  that  says  "This  is  what  you  do." 
Nobody  knows  what  to  do  now;  it's  too 
complicated.    But  anyway,  if  somehow 
the  office  of  planning  could  come  down 
with  something  that's  a  real  neat 
package  of  "VThat  do  I  do?*'  it'd  be 
great.    That's  what  people  are  looking 
for.    They  have  to  get  their  hooks 
into  something,  and  thoy  don't  know  where 
to  hook  into,  and  that's  got  to  come 
pretty  soon  or  the  whole  spirit  there  is 
cibout  planning  is  going  to  fade.32 

To  be  sure,  planning  personnel  have  conducted  numerous 
in-service,  training  sessions  about  planning.    v?hon  oeople 
leave  the  session,  they  scorn  to  think  themselves  ill^ 
equipped  to  begin  planning  on  their  own.    This  problem 
should  abate  as  ESED  #2  develops  its  own  plan  for  planning. 


Section  2:    Attitudinal  Data 

The  data  presented  in  this  section  aire  organized  as 
follows: 

A.  Mobilization  of  Organizational  Plp.nning 

1.  operational  Impact  of  Organi^^itional 
Planning 

2.  Role  of  Planning  Unit 

3.  Development  of  Evaluation  Tr-chniaues 

B.  Top  Management  Support  for  PI  running  ' 

C.  Credibility  of  the  Pl^inning  Process 

The  data  presented  under  Mobilization  of  Organiza- 
tional Planning  pertain    to  the  riVi's  training  goals  eight 
through  13.    These  items  relato  specifically  to  the  opera- 
tionalization  of  planning  in  the  experimental  agencies 
which  the  AIU^.  training  goals  imply. 

In  addition,  ^^'o  have  also  presenter]!  data 
indirectly  related  to  the  A?1A's  training  goals  but 
directly  related  to  the  establishment  and  achievement  of 
these  goals.    Top  Management  support  for  planning  and 
the  establishment  of  a  credible  planning  process  are 
viewed  here  as  essential  causal  variables.    Thc:^  degree  to 
which  these  variables  exhibit  change  indicates  the 


^^Ibid. 
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af fectiveness  or  lack  of  effect  of  the  training.  These 
varicibles  should  also  foreshadow  the  course  of  future 
planning  developments  within  these  organizations. 


A,    Mobilization  of  Organizational  Planning 

We  have  divided  this  sub-category  into  three  parts, 
each  explained  scpcirately. 

1.    Operational  Impact  of  Organizational  Planning 

The  ten  items  presented  here  concern  the  assignment 
of  responsibilities  and  the  development  of  performance 
standards  essential  to  any  organizational  planning  effort. 

The  first  six  items  are  content  analysis  categories. 
Because  the  first  three  items  are  content  categories  of 
the  same  interview  q  zBtion,  the  comments  apply  to  all 
three.    The  same  is  true  of  the  next  two  items.    The  last 
four  items  are  taken  from  the  questionnaire. 

1.  Define  standards  of  performance  for 
key    administrators  ^ 

2.  Specify  task  completion  dates  and  action 
assignments 

3.  Assign  responsibilities  to  subordinate  units 

Interview  Question:    What  do  you  think  you 
will  obtain  (have  obtained)  from  the 
iVIA's  trainin^  program? 

Range  of  Scale  Possibilities:     (1)  no  value 
to  (7)  maximum  value. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:    El  and  E2  only. 

4.  Need  for  Performance  Standards 

Interview  Question:    Have  performance 

standards  been  established  for  your 
subordinates  based  on  the  objectives 
in  your  division  plan? 

Range  of  Scale  Possibilities:    Extent  of  Use: 
(1)  none  at  all  to  (7)  definitely. 
Need  for:     (1)  no  value  -  should  not 
be  used  at  all  to  (7)  should  be  used 
O  much  more. 

ERIC 
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Points  of  Time:  T4, 
States:    El,  E2,  CI, 

5.  Performance  Standards  -  Extent  of  Use 

6.  Existence  of  Performance  Reviews 

Interview  Question:    Do  you  have  regular 

performance  reviews  with  your  subor* 
dinates? 

Range  of  Scale  Possibilities:     (1)  still  not 

used  to  (7)  regular  performance  reviews 
held. 

Points  of  Time:  T4. 
States:    El,  E2,  CI. 

The  Questionnaire  items  are: 

7.  My  organization's  policy  statements  are  clear. 
Points  of  Time:    Tl,  T2,  T3,  T4* 

8.  My  organization's  performance  standards  are 
understood . 

Points  of  Time:    Tl,  T2,  T3,  T4. 

9.  Good  ways  are  used  to  let  me  know  how  I  can 
improve  my  performance.  — — ^— . 

Points  of  Time:    Tl,  T2,  T3,  T4. 

10.  7  understand  what  results  must  be  produced  to 
dtc^  leve  the  stared  objectives  of  this  organi- 
zation^ 

Points  of  Time:    T3,  T4 


149 


Item  _1 :  Define  standards  of  performance  for  key  administrators 


Fall.  1970 

Spring.  1971 

Fall.  1971 

Spring,  1972 

S  E2 

El      S  E2 

E^      §  E2 

N 
2 

N 
2 

N 
9 

N 
5 

N  N 
3  6 

N  N 
6  9 

Kruska 

L-Wallis  One-Way  Analysis  of  Variance 

H=  0.600 
Sie.=  NS 

H=  0.360 
Sie.=  NS 

H= 1.066 
Sie.=  NS 

H=  0.003 
SiE.=  NS 

Binomial  Test  of  Proportions 

P=0.644 
Si2.=  NS 

P=0. 028 
Sie.=  .05 

P=0.198 
Sie.=  NS 

P=0.104 
Sie.=  NS 

Fall  1970         to               Soring  1972 

El  S 

E2         §  E2 

N 
2 

N 
6 

N 
2 

N 
9 

Kruska 

.-Wallis  One-Way  Analysis  of  Variance 

H=  1.777 

H=  0.680 

Sig.=  NS 

Binomial  Test 

of  Proportions 

P=  0.000095 
Sig.=  .001 

P=  0.000001 
Sig.=  .001 

ERIC 
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Item   2_:    Specify  task  completion  dates  and  action  assignments 


Fall.  1970 

SprinK.  1971 

Fall.  1971 

Spring,  1972 

5  E2 

E^    S  E^ 

Ej      §  E2 

Ej    a  E2 

N 
1 

N 
3 

N 
9 

N 
8 

N 
3 

N 
5 

N 
6 

N 
4 

Kruska 

l-Wallis  One-Wai 

f  Analysis  of  Variance' 

H»  1. 800 
Sia.=  NS 

H=0.750 
Sia.«  NS 

H=  0.200 
Sie.=  NS 

H=  3.681 
SiB.=  NS 

P=0.316 
Sia.-  NS 

P=0.702 
.    Sia.=  NS 

P=0.506 
SiB.»  NS 

P=0.152 
Sia.»  NS 

Fall  1970        to              Sorine  1972 

El  5 

E2          5  E^ 

N 
1 

N 
6 

N 
3 

N 
4 

Kruska 

l-Wallis  One- Way  Analysis  of  Variance 

H=  1.5625 

H=  3.1250 
Sip.=  N£ 

Binomial  Test  of  Proportions 

P=  0.000001 
Sig.«  .001 

P=  0.078 
Sig.=  NS 
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Item  _3  :  Assign  responsibilities  to  subordinate  units 


Fall. 

1970 

Spring.  1971 

Fall, 

1971 

Spring, 

1972 

^1  ^ 

^2 

^1  ^ 

^1  « 

i  ^2 

N 

N 

N 

N 

N 

N 

N 

N 

2 

3 

6 

4 

7 

8 

7 

9 

Kruska 

1-Wallis  One-Way  Analysis  of  Variance 

H=  0. 

083 

H=l. 

136 

H=  0. 

000 

H=  2 

.042 

Sie.= 

NS 

Sie.= 

:  NS 

Sie.= 

:  NS 

Sig.= 

=  NS 

Binomial  Test  of  Proportions 

P=0.780 

P=0. 

386 

P=0.872 

P=0 

.450 

SiK.= 

:NS 

.  Sig.= 

=  NS 

Sig.= 

:  NS 

Sig.= 

=  NS 

Fall  1970  to 

Spring  1972 

^1 

1 

^  ^1 

E2  5 

N 

N 

N 

N 

2 

7 

3 

9 

Kruska! 

L-Wallis  One-Way  Analysis  of  Variance 

H=2.142 

H=  4 

.521 

NS 

Sie.= 

.05 

Binomial  Test  of  Proportions 


P=  0.000005 

P=  0.000001 

Sig.=  .001 

Sig.=  .001 
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Item  4     Need  for  Performance 
Standards. 


Item  5 


Performance  Standards 
Extent  of  Use. 


prine.  1972 

S] 

orinp.  1Q77 

El  a 

Ej    5  E2 

jfay  An{ 
ft  of  I 

El    S  Ci 

E2    5  Cj 

El    5  E2 

N 
9 

N 

N 

10 

N 

11 

N 
9 

N 

10 

N 
7 

N 

11 

N  N 
10  11 

N 
7 

N 

10 

H=  0.831 
Sig.=  NS 

H=  0.600 
Sig.=  NS 

H=2.281 
Sig.=  NS 

.lYSis  ot  vai 
H=  0. 820 

Sig.=  NS 

pance 
H=  0. 001 

Sig.=  NS 

H=0.609 
Sig.=  NS 

Binomial  T«« 

P=  1.000 
Sig.=  NS 

P=  1.000 
Sig.=  NS 

i'=1.000 
Sig.=  NS 

P=  0.000 
001 
Sig.=.001 

P=  1.000 
Sig.=  NS 

P=0.000 
Sig.=.  001 

Item  6      Existence  of  Performance  Reviews 


Krus 


fiprifig  1079 


Ej        §  Cj 


N 
9 


N 
11 


E2         5  Cj 


N 

10 


N 

11 


Ej  §  E2 


N 
9 


N 

10 


cal-Wallis  One-Wav  Analysis  of  Varianrp 


H=7.274 
Sig.=  .01 


P=  1.000 
Sig.=  NS 


H=0.209 
Sig.=  NS 


H=  3.081 
Sig.=  NS 


Binomial  Test  of  Proportions 


P=  1.000 
Sig.=  NS 


P=  1.000 
Sig.=NS 
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Item       10        I  understand  what  results  must  be 
produced  to  achieve  the  stated 
objectives  of  this  organization. 


Fall,  1971 

Spring,  1972 

Experimental  SEDil 

N 

$8 

X 

5.397 

so 

1.457 

N 

40 

X 

5.075 

SD 

1.163 

Eiqperlaentftl  SED«2 

45 

4.688 

L.427 

4o 

4.325 

1.456 

Control  SED  «1 

67 

4.955 

1.353 

60 

4.800 

1.493 

Control  SED  #2 

61 

4.803 

1.661 

65 

4.615 

1.276 

Total 

 « 

241 

205 

IVro  Nay  Analysis  of 
Variance 

F 

Simif. 

Experimental  SED  #1 

]ol. 

1.649 

NS 

W/Control  SED  *1 

Row 

3.720 

NS 

Eiqperimental  SED  #1 

Col. 

7.693 

.01 

w/Control  SED  *  2 

low 

1.802 

NS 

Experimental  SED  #2 

Col. 

1.673 

NS 

w/Contxol  SED  *1 

Row 

3.415 

nS 

E]q>erimental  SED  #2 

Col. 

1.801 

NS 

w/Control  SED  »2 

tow 

6.972 

MS 
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Analysis  of  items  1,  2,  and  3  reveals  no  change 
between  the  ExpcrimGntal  States  sustained  during  the 
period  Fall,  1970  to  Spring,  1972.     In  only  one  case  did 
ona  item  gain  significantly  greater  emphasis  over  time 
within  a  State;  E2's  top  managers  adopted  Item  ^^3,  assign- 
ing more  responsibilities  to  subordinate  units  in  the 
Spring  of  1972  than  they  did  before  training* 

Within  all  states ,  with  one  exception,  a  significant 
increase  appeared  in  the  number  of  top  managers  who  men- 
tioned that  the  AI1A  training  goals  were  obtained  as  a 
result  of  training.    While  E2  would  be  judged  to  have 
shown  no  effect  of  training  on  Item  #2,  specifying  task 
completion  dates  and  action  assignments,  all  other  items 
indicate  positive  effects  of  training. 

The  next  three  items  assess  the  use  of  performance 
standards  and  reviews  in  the  experimental  organizations. 

Item  #4,  concerning  the  need  for  performance 
standards  shows  that  the  top  managers  of  the  Experimental 
and  Control  States  all  consider  performance  standards 
important.    Item  #5,  indicates  more  widespread  use  of 
performance  standards  in  both  the  Control  and  State  E2 
than  in  State  El.    This  finding  corresponds  with  our  earlier 
finding  that  job  descriptions  were  more  widely  used  in  E2 
than  in  El. 

Actual  performance  reviews  are  given  greater 
emphasis  in  El  than  in  the  Control  as  indicated  in  Item  #6, 
but  no  significant  differences  exist  on  this  point  between 
States  El  and  E2. 


Item  #7s     Fall  1970  to  Spring  1971 

The  seventh  item  in  this  section,  My  organization' s 
policy  statements  are  clear,  showed  no  ^significant  differ- 
ences  in  the  El  comparison  cither  over  time  or  holding 
time  constant.     In  short,  no  differences  between  the  States 
and  no  effect  of  training  were  shown.    The  E2  comparison 
indicated  significant  differences  which  were  attributable 
to  training  and  were  produced  by  a  positive  effect  in 
State  E2;  when  time  was  held  constant,  no  difference 
existed  between  these  States. 


Item  #7:     Spring  1971  to  Fall  1971 

The  fiMA  program  had  no  effect  in  either  State  dur- 
ing this  period.    We  found  no  statistical  difference  in 
how  El  or  Cl  perceived  their  organization's  policy  state- 
ments; neither  did  an  analysis  of  changes  between  Spring 
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and  Fall  1971  yield  any  real  changes.    Policy  statements 
wore  definitely  clearer  in  CI  than  in  E2,  even  though  CI 
showed  a  minor  decrease  while  E2's  understanding  rose 
slightly.    These  negligible  changes  were  not  sufficient 
to  produce  T2-T3  differences  of  any  significance. 


Item  #7;    Fall  1971  to  Spring  1972 

There  were  also  no  effects  of  JuMA  training  during 
T3-T4.    An  incroasG  in  El's  perception  of  statement 
clarity  can  be  compared  with  a  corresponding  decrease  in 
C2  to  produce  a  significant  difference  betv^een  them  but 
neither  changed  sufficiently  over  time  to  conclude  that 
training  had  any  impact.    E2  and  CI  remained  essentially 
static  between  Fall  1971  and  Spring  1972  although  a 
difference  still  existed  between  the  States — CI  continued 
to  consider  its  policy  statements  more  understandable 
than  did  E2, 


Item  #7;     Fall  1970  to  Spring  1972 

Examining  organizational  policy  statements,  v;e  found 
that  El  and  E2  did  not  change  significantly  betv;een  Pall 
1970  and  Spring  1972.    A  slight  increase  was  recorded  in 
El  while  E2  stayed  at  approximately  the  same  level.  No 
difference  existed  between  El  and  CI  but  continued  for  E2 
and  CI;  E2  still  thought  less  of  what  their  organization 
wrote  than  CI  thought  of  their  documents.    Taking  a  com- 
bined Y1-Y2  perspective,  then,  wg  conclude  that  the  Air 
training  program  had  no  effect  on  the  clarity  of  either 
r.l  or  E2's  policy  statements. 


Item  #8:    Fall  1970  to  Spring  1971 

Item  #8,  My  organization's  performance  standards 
are  understood «  showed  no  effects  attributeisle  to  training 
in  El  from  Fall  1970  to  Spring  1971.    The  States  differed) 
however,  in  that  the  Experimental  State  perceived  its 
perfomance  standards  as  better  understood  than  was  the 
case  in  the  Control  State,    rftatc  F2  evidenced  training 
effects  and  proved  to  be  different  from  the  Control.  In 
this  case,  however,  the  effect  of  training  was  to  reduce 
the  extent  to  which  performance  standards  wore  understood. 
The  differences  between  the  States  was  that  tho  Control 
State  perceived  that  their  standards  were  better  understood 
than  those  of  Experimental  State  E2. 
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Item  #8t     Spring  1971  to  Fall  1971 

li  look  at  E2  during  this  period  indicates  that, 
v/hile  they  continued  to  understand  their  perfonnance 
standards  much  less  than  Cl,  no  significant  change 
occurred  within  the  State.    Therefore  E2  vras  not  affected 
by  the  7\?1A  program  in  teanns  of  this  aspect  of  performance 
standards.    Something  different  occurred  in  Fl,  where  the 
organization  changed  significantly  between  Spring  and  Fall 
1971,  showing  a  net  reduction  of  comprehension  of  current 
standards  of  performance.    Because  El*s  understanding  was 
declining  faster  than  the  Control  State's,  as  v/ell  as  being 
more  unstable,  a  negative  effect  of  AMA  training  was 
revealed. 


Item  #8;     Fall  1971  to  Spring  1972 

El  stabilized  between  T3  and  T4,  showing  no  internal 
change  or  external  differences  with  either  Cl  or  C2.  Under- 
standing of  performance  standards  remained  about  the  same 
in  E2  also  although  it  was  still  lower  than  in  Cl  as  well 
as,  for  the  first  time,  C2.    Since  no  State  changed  between 
Fall  1971  and  Spring  1972,  we  concluded  that  I\nVi  training 
had  no  impact  on  El  or  E2. 


Item  #8s     Fall  1970  to  Spring  1972 

An  evaluation  of  V7hat  effect,  if  any,  the  AI'4A 
training  program  exerted  on  E1/E2  understanding  of  perfor- 
mance stcindards  depends  on  a  direct  comparison  betv;een  data 
gathered  before  training  began  (Pall  1970)  and  data 
gathered  after  training  had  ended  (Spring  1972).     In  this 
way  we  can  deteirmine  if  effects  observed  during  interim 
periods  were  permanent  or  temporary.    As  far  as  SI  is 
concerned  any  negative  effects  attributed  to  TdiA  training 
were  temporary.    T*7hilc  there  was  an  overall  decline  in 
how  much  this  organization  knew  about  its  performance 
standards.  El  did  not  decline  as  fast  as  Cl;  no  significant 
difference  existed  between  them.    Therefore,  a  slight 
positive  effect  can  be  credited  to  PJ'I.  training  in  El. 
It  is  also  clear  that  the  negative  impact  originally 
indicated  T1-T2  for  E2  faded  with  time.    VThile  E2  began 
and  ended  lower  titan  any  other  State  and  even  dropped  on 
a  T1-T4  comparison,  this  drop  was  not  so  severe  as  the 
decline  in  Cl  nor  was  it  so  unstable.    ^*^o  can  show  a 
positive  effect  of  A21A  training  of  E2's  understanding  of 
performance  standards. 
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Itcn  #9i    Fall  1970  to  Spring  1971 

Training  had  a  positive  effect  on  E1/E2  opinions 
of  the  extent  to  vhich  Good  ways  are  used  to  let  me  know 
how  I  can  improve  ny  yerTbrmance  during  this  initial 
period.    To  be  sure,  in  comparison  to  E2  better  \fays  were 
characteristic  of  CI  both  before  and  after  training.  El' 
remained  higher  than  either  E2  or  CI.    But  an  analysis  of 
how  they  changed  over  time  reveals  that  CI  declined  more 
sharply  than  did  El  or  E2  between  Pall  1S70  and  Spring 
1971.    Assuming  that  without  AMA  training  the  Experimental 
States  V70uld  have  experienced  a  similar  rate  of  decline, 
we  attribute  positive  effects  to  El  and  E2. 


Item  H9:     Spring  1971  to  Fall  1971 

There  were  statistical  differences  bef/zeen  CI  and 
E1/E2  throughout  this  period;  these  differences  consisted 
in  CI  continuing  to  feel  more  rositive  than  did  E2  about 
its  means  of  showing  organization  members  hov;  they  might 
improve  performance;  but  CI  remained  more  critical  of 
themselves  than  did  El.    Neither  El  or  E2  expe;  lenced  any 
significant  change  as  a  result  of  AHA  training. 


Item  #9;    Fall  1971  to  Spring  1972 

This  comparison  adds  C2  to  our  analysis  and  strengthens 
our  conclusion  that  performance-related  communication  was 
not  increased  as  a  result  of  A^JA  training  between  Pall  1971 
and  Spring  1972.    El  and  CI  showed  greater  approval  of 
their  ways  to  transmit  information  on  individiaal  performance 
than  did  C2  or  E2.    But  looking  at  Cl/Cl,  E1/C2,  E2/C1,  and 
E2/C2  over  time,  no  significant  changes  are  shown  in  El  or 
E2. 


Item  #9;    Fall  1970  to  Spring  1972 

A'lA  training  had  no  overall  impact  on  the  ways  in 
which  performance  data  was  communicated  in  El  or  E2" 
Neither  Experimental  State  significantly  changed  its  self- 
appraisal  on  this  issue  between  Fall  1970  (before  training) 
and  Spring  1972  (a  year  after  training  had  ended) .  There 
remained  a  significant  difference  between  E2  and  CI  but 
this  is  not  relevant  to  analyzing  training  effects  unless 
some  change  was  also  registered  over  time  within  the  States. 


Item  #lOt    Pall  1971  to  Spring  1972 

Analysis  of  Item  #10,  I  understand  what  results  must 
rnir  be  produced  to  achieve  the  stated  objectives  of  this  organl- 

tj^;  zation  is  confined  to  T3  and  T4  although  -i.t  includes  four 
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states:     2  Experimental  States  and  2  Control  States,  The 
only  significant  comparison  involves  El  and  C2  between 
Pall  1971  and  Spring  1972,  and  it  roveals  a  significant 
change  over  time*    An  understanding  of  what  results  must 
be  produced  has  dropped  in  both  States •    However,  this 
change  is  too  weak  to  produce  any  important  differences 
between  El  and  C2. 


DATA  SUMMARY 


Mobilization  of 
Organizr.^^ional  Planning 


Fall 

,  1970  -  Spring,  1972 

K4PACT  OP  TRAINING 

Item 

Type  of  Data 
CONTENT 

Positive 
Effect 

No  Negative 
Effect  Effect 

1 

Define  standards  of  per- 
formance for  key  admin- 
istrators 

E1,E2 

2 

Specify  task  completion 
dates  and  action  assign- 
men  t.s 

E1,E2 

3 

Assign  responsibilities 
to  subordinate  units 

E2 

Fl 

4 

Need  for  Performance 
Standards 

Insufficient 

5 
6 

Performance  Standards — 
Extent  of  Use 

Existence  of  Performance 
Reviews 

dat?.  (T4  only) 
on  which  to  base 
a  response* 

QUESTIONNAIRE 

7 

My  organization's  policy 
statements  aro  clear* 

E1,E2 

8 

My  organization's  perfor*^ 
mance  standards  ere 
understood. 

E1,E2 

9 

Good  way?  are  used  to  let 
me  know  how  I  can  Improve 
my  performance • 

E1,E2 

10 

I  understand  what  results 
must  be  produced  to  achieve 
♦ihe  stated  objectives  of 
this  organization*  . 

E2  El 
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There  were  several  effects  of  training  on  the  mobili- 
zation of  organizational  planning.    One  was  negative;  three 
were  positive;  ten  comparisons  showed  no  effect.  Under- 
standing of  expected  results  (Item  #10)  declined  more 
severely  in  El  than  it  did  in  one  of  the  Control  States 
against  which  this  Experimental  State  was  conpared.  E2 
assigned  responsibilities  (Item  #3)  and  comprehended  its 
performance  standards  (Iteui  #8)  better  as  a  result  of  train- 
ing.   VJhile  El  did  not  improve  its  assignment  process, 
the  staff  knew  more  about  their  standards  of  performance. 

Training  did  not  alter  existing  differences  between 
the  States,  however.    When  questionnaire  items  are  examined. 
El  appears  most  effective  in  planning  mobilization  while 
E2  was  least  effective;  in  fact  E2  was  less  effective  than 
the  Control  State  which  had  recently  entered  the  I^ih  program 
These  relative  standings  are  not  consequences  of  training; 
they  were  the  same  before  and  after  training. 

Throughout  the  periods  covered  by  this  evaluation, 
the  extent  to  which  performance  standards  and  expected 
results  were  understood  declined  in  both  Experimental  States 
On  the  other  hand,  the  clc  ^ity  of  policy  statements  and 
means  of  communicating    how  to  improve  performance  remained 
comparatively  stable  between  the  Fall  of  1970  and  the 
Spring  of  1972. 

The  absence  of  data  is  due  to  the  previously  men- 
tioned fact  that  Items  #4-6  were  administered  in  T4  only 
and  thus  we  have  insufficient  data  to  attribute  training 
effects  to  ?\I1A.  VJe  can  say  that  performance  standards  seem 
to  be  more  fully  developed  in  El  than  in  CI  during  the 
Spring  of  1972.  Whether  this  was  one  effect  of  spending 
several  weeks  in  Hamilton  is  uncertain. 

But  deta  on  the  other  items  was  sufficient  for  the 
research  team  to  assess  the  impact  of  the  program.    On  the 
basis  of  this  evidence,  we  conclude  that  training  had  no 
effect  on  most  of  the  variables  associated  with  the 
mobilization  of  organizational  planning. 
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2.    Role  of  the  Planning  Unit 

The  seven  items  presented  here  address  the  specific 
activities  of  the  planning  units  established  in  both 
Experimental  States  prior  to  AIIA  training.    The  items  are 
also  conceptually  related  to  the  Mil^  training  goal  of 
nurtiaring  supplementary  planning  efforts.    The  first  six 
items  are  content  categories  of  the  same  interview  question , 
the  last  item  appeared  in  the  questionnaire. 

1.  Awareness  of  need  to  evaluate  our  programs 

2.  Available  to  answer  planning  questions 

3.  Writing  guidelines  for  plan  development 

4 .  Reviewing  and  refining  plans 

5.  Provides  leadership  in  the  implementation 
of  planning 

6.  Provides  in-service  training  in  planning 

Interview  Question:    How  has  the  Planning 
unit  helped  you  to  plan? 

Range  of  Scale  Possibilities;     (1)  no  help 
to  (7)  great  help 

Points  of  Time:    I'S,  T4. 

States:    El,  E2, 

The  Questionnaire  item  is: 

7.  The  planning  unit  has  been  helpful  to  me. 


Points  of  Time:    T3,  T4. 
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Item 


Role  of  Planning  Unit  -  Awareness 
of  need  to  evaluate  our  programs. 


Kruskal 


Fall, 

1Q71 

fipringi 

t  1972  1 

^1 

S  E2 

»  ^2 

N 

N 

N 

N 

0 

0 

2 

3 

H=  0.000 


Sig.= 


NS 


ATia lysis  nf  Variance 


H=  0.750 
Sig.=  NS 


P=  0.000 

P=  0.924 

Sig.=  NS 

Sig.=  NS 

Fall. 

1971      to    Snrinc.  197? 

\  ' 

Ej  S 

^2 

N 

N 

N 

N 

1 

2 

0 

3 

Kruskal -Wani«!  OnP-Wav 


H=  1.500 


Sig.=  NS 


H=  0.000 

Sig.=  NS 


Binomial  Test  of  Prnpnrti  nti?; 


P=  0.068 
Sig.=  NS 


P=  0.000 
Sig.=  NS 


ERIC 


166 


Role  of  Planning  Unit  -  Available 
to   \nswer  planning  questions. 


Pflllj  1Q71 


Ej    S,  E2 


N 
6 


Kruskal 


N 


S  E- 


N 
8 


N 

9 


H=  4.591 

H=  7.787 

Sig.=  .05 

Sig.=  .01 

Binomial  Ta«»  v^^j^^*.i^^^^ 


P=  0.924 
Sig'=  NS 


P=  0.774 
Sig.»  NS 


Kruskal 


Fall. 

1971 

to    Snrinff.  197 

Ej  § 

N 

N 

N 

N 

6 

8 

7 

9 

-Wall  is 

Onft-Wav 

H=  0.600 

H«  3.050 

NS 

NS 

lance 


P=  0.026 

P«  0.028 

Sig. a  .05 

Sig..  .05 

167 


Role  of  Planning  nnii:  -  Writing  guidelines 
for  plan  development. 


Kruskal 


1Q71 

1  1972 

^1 

5  E2 

i  E2 

N 

N 

N 

N 

6 

3 

6 

4 

H=  3.266 

H=  6. 

545 

Sig.= 

NS 

Sig.= 

.02 

lance 


p=  0.038 
Sig.=  NS 


0.152 

Sig.=  NS 


Fall. 

1971      to    Soring.  197 > 

Ej  § 

N 

N 

N 

N 

6 

6 

3 

4 

-Wall is  Onft-Wav 

c   nf  Vo-r» 

H=  0. 006 

H=  4.500 

Sig.= 

NS 

Sie.= 

.05 

Rinomial  Test  of  Prnnort-ioTK; 

P=  0.377 

P=  0.150 

Sig.= 

NS 

Sig.= 

NS 

lance 
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Role  of  Planning  Unit  -  Reviewing 
and  refining  plans. 


Kruska 


Spring 

i-m2  , 

^1 

«  E2 

^1  ' 

1  E2 

N 

N 

N 

N 

1 

5 

5 

6 

H= 0.085 

H.  4.033 

Sig.= 

NS 

Sig.= 

.05 

P«  0.0002 
Sig..  .001 


pa  0.964 
Sig.«  NS 


Kruskal 


Falli  1971      to    Snrina.  107 

N  N 

1  5 

N  N 

5  6 

-Wallis  Or.«_Wfly 

H=  0.771 


Sig.a  NS 


H»  0.675 


Sig.-  NS 


Binopial  Tast  of  Pynpftt.«-i/>«c 


P=  0.000118 
Sig.=  .001 


P=  0.171 
Sig.«  NS 
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Role  of  Planning  Unit  ~  Provides 
leadership  in  the  implementation 
of  planning. 


Kruskal 


1071 

Spring 

,  1972 

^1 

5  E2 

r  E2 

N 

N 

N 

N 

5 

6 

7 

5 

H=  1. 

408 

H=3.487 

Sig.= 

NS 

Sig.= 

NS 

'lance 


P=  0.964 

P=  0.098 

Sig.=  NS 

Sig.=  NS 

Fall. 

1971      to    Snrini?.  197! 

Ej  § 

N 

N 

N 

N 

5 

7 

6 

5 

-Wani«:  nni>_Wflv 

Analvcic  nf  Vjjt 

H=  1. 

114 

H=  2. 

408 

SiK.= 

NS 

3ie.= 

NS 

P=  0.]41 

P=  0.366 

Sig.=  NS 

Sig.  =  NS 
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Role  of  Planning  Unit  -  ProvidQs 
in-service  training  in  planning. 


Kruskal 


,  Fall, 

1Q71 

^-L272  .. 

^1 

S  E2 

i  E2 

N 

N 

N 

N 

3 

4 

2 

2 

H=  1.125 

Sig.=  NS 


Analysis  nf  Vyiance 


H=  2.400 
Sig.=  NS 


Rinnmial  Toc«- 


P=  0.964 
Sig.=  NS 


P=  0.678 
Sig.=  NS 


Fall.  1971     to   Siring.  197 

Ej    §  E2 

N 

3 

N 

2 

N  N 

4  2 

-WalliS  One-W.1v  Anfllvcic  Var. 

H=  0.750 
Sig.=  NS 

H=3.428 
Siff.=  NS  ! 

Binomial    T^^t   »r  Pr-nppy^^pn 


P=  0.377 
Sig.s  NS 


P=  0.167 
Sig.=  NS 
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I  tern    7          The  planning  unit  has  been  helpful 
to  me.  .  


< 

Fall,  1971 

Spring, 
T 

1972 

4 

Experinental  SED#1 

N 

68 

X 

41867 

SD 

1.930 

N 

38 

4 

X 

.631 

SD 

1.323 

Experiaental  SED*2 

45 

3. "644 

1.734 

40 

3 

.750 

1.497 

Contxol  SED  «1 

67 

3.746 

1.778 

53 

3 

.849 

1.472 

Control  SED  *2 

3.737 

1.731 

64 

3 

.593 

1.466 

Total 

241 

195 

IVro  Wi^  Analysis  of 
Variance 

Experiaental  SED  #1 

F 

Siimif. 

"fil. 

0.082 

NS 

W/Control  SiiD  «1 

Row 

16.908 

• 

001 

E)qperi»ental  SED  «1 

Col. 

0.712 

NS 

w/Control  SED  *  2 

Row 

23.185 

• 

001 

Ejqperiaental  SED  #2 

Col. 

0.198 

NS 

w/Control  SED  «1 

Row 

0.184 

NS 

E^erinental  SED  #2 

Col. 

0.007 

NS 

w/Control  SED  «2 

low 

0.019 

MS 

Items  #1  through  #6  were  asked  only  in  Fall  1971 
and  Spring  1972  and  were  framed  to  compare  the  roles  of 
the  planning  units  in  the  Experimental  States,  Because 
this  question  was  not  asked  before  training  we  cannot 
make  any  judgments  regarding  the  impact  of  training  on  the 
activities  of  these  planning  units.     It  should  be  pointed 
out  nonetheless  that  both  planning  units  were  established 
just  before  AMA  training. 

The  role  of  the  planning  units  in  developing  staff 
av/areness  of  the  need  to  evaluate  programs   (Item  #1),  to 
provide  leadership  in  the  implementation  of  planning  (Item 
#5),   and  to  offer  in-service  training  (Item  #6),  showed  no 
differences  between  the  states  or  within  them.     An  analy- 
sis of  the  number  of  people  who  responded  indicates  that 
neither  planning  unit  is  seen  as  much  affecting  evaluation 
in  the  planning  process  or  in-service  training.  Many 
managers  did  find  that  the  units  gave  helpful  leadership 
in  the  planning  process. 

An  analysis  of  the  role  of  planning  units  in  assist- 
ing the  planning  process  shows  that  top  managers  in  El 
viewed  their  unit  as  being  more  helpful  than  did  top  man- 
agers of  E2. 

Item  #7-  rhows  significant  differences  between  the 
two  states.     El  is  seen  as  being  more  helpful  than  E2  in 
Fall  1971  and  Spring  1972.     Both  states  showed  signifi- 
cantly increasing  awareneis  of  the  assistance  given  by 
their  planning  units  In  tie  planning  process. 

Item  #3  showed  ..har  managers  in  El  saw  t  le  written 
guidelines  for  planning  as  being  more  helpful  t^^n  those 
in  E2.     This  item  also  indicates  that  over  time  \he  planning 
unit  in  E2  appeared  less  ^.elpful  in  this  regard  than  was 
the  case  in  Fall  1971.     Th.'.s  probably  reflects  the  develop- 
ment of  a  plan  for  planning  in  El  which  wan  not  produced 
in  E2. 

Item  #4  indicated  a  dramatic  change  in  the  helpful- 
ness of  the  planning  unit  in  reviewing  and  refining  plans 
in  El,  by  contrast  to  E2.     The  planning  unit  in  El  appar- 
ently increased  its  activity  in  this  area  significantly. 

Item  #7;     Fall  1971  to  Spring  1972 

Item  #7,  The  planning  unit  has  been  helpful  to  me, 
was  used  to  compare"  changes  in  El  and  E2   (as  contrasted  to 
CI  and  C2)   between  Fall  1971  and  Spring  1972.  Analysis 
reveals  that  persons  in  El  considered  their  planning  unit 
significantly  more  he\pful  than  those  in  CI  or  C2,  though 
impressions  in  no  stuZe  changed  over  time.     The  planning 
unit  in  E2,  while  stc. ^istically  similar  to  the  Control 
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States,  was  viewod  as  much  less  helpful  by  persons  in 
their  organization  than  was  its  counterpart  in  El.  A 
visual  inspection  of  the  tables  demonstrates  that  F2  had 
lower  ratings  than  any  State  in  T3  and  lower  than  any 
except  C2  in  T4.    To  repeat ,  however,  all  these  differences 
are  statistically  insignificant. 


DATA  surmRY 
Role  of  the  Planning  Unit 


^all,  1970  -  Spring/  1972  IMPACT  OF  TRAI^^^!G 


Item 

Typo  or  uaua 
COl^TEWT 

Positive 
Effect 

No 
Effect 

Negative 
Effect 

1 

Awareness  of  need  to  evalu* 
ate  our  programs 

E1,E2 

2 

Available  to  answer  plan- 
ning questions 

E1,E2 

3 

Writing  guidelines  for 
plan  development 

El 

E2 

4 

Reviewing  and  refining 
plans 

E1,E2 

5 

Provides  leadership  in  the 
implementation  of  planning 

E1,E2 

6 

Provides  in-service  trainr 
ing  in  planning 

QUESTIONNAIRE 

E1,E2 

7 

The  planning  unit  has  been 
helpful  to  me. 

E1,E2 

The  ssr vices  provided  by  Experimental  State  planning 
units  to  other  offices  and  persons  vrithin  their  organiza- 
tions were  not  Jjnoroved  by  the  AMT.  training  program.  When 
writing  guidelines  for  planning  (Item  #3)  is  considered, 
sexrvices  were  adversely  affected.    Between  Pall  1971  and 
Spring  1972,  E2  felt  that  theii  planning  unit  had  become 
less  helpful  in  this  activity. 

The  reader  is  again  advised  to  treat  this  information 
cautiously     Absence  of  a  control  group  and  pro-training 
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data  for  comparative  purposes  do  not  strengthen  our 
findings. 

However^  short  of  attributing  training  effects^  we 
can  make  observations  and  point  to  certain  differences. 
Neither  Experiirental  State  was  more  aware  than  the  other  about 
any  of  these  items  except  during  T3.    In  assessing  their 
planning  unit's  assistance  to  them  then^  E2  mentioned 
"reviewing  and  refining  plans"   (14)  more  than  El;  although 
the  data  does  not  tell  us  if  the  observations  were  more 
favorable  or  unfavorable. 

W.at  people  thought  about  planning  services  is 
covered  by  the  analysis  of  variance.    No  difference  existed 
betvreen  the  States  on  half  of  the  content  items;  but  El's 
planning  unit  was  viewed  more  favorably  or:  the  other  half. 
This  Experimental  State  felt  that  they  received  more  aid  in 
finding  answers  (Item  ^2),  developing  plans  (Item  -^3),  and 
reviewing  plans^  than  did  their  counterparts  in  E2.  This 
advantage  remained  relatively  stable  over  time  except  in 
regard  to  guidelines  for  plan  development;  E2  endured  such 
a  decline  that  negative  training  effects  v/ere  assigned. 

The  summary  questionnaire  item  tnat  elicited  overall 
impressions  of  how  helpful  a  planning  unit  had  been  also 
revealed  no  training  effects.    It  did  indicate  that  E2's  plan- 
ning unit  was  considered  more  helpful  than  units  in  E2  or 
Clf  even  though  this  difference  was  not  significant.  The 
dc^ta  also  showed  that  persons  within  E2  had  achieved  greater 
consensus  of  opinion  about  their  unit  between  T3-T4. 


3.    Development  of  Evaluation  Techniques 

This  section  contains  nine  items  specifically  con- 
cerned with  the  use  of  evaluation  techniques  in  the  planning 
process.    The  first  item  represents  one  of  T^Ui*&  initial 
training  goals.    The  next  five  items  are  directly  related 
to  the  goal  and  to  specific  evaluation  techniques  which  the 
states  may  or  may  not  be  using.    The  last  three  items  are 
excerpted  from  the  questionnaire  and  are  related  to  the 
AMA's  goal  for  evaluation  methodology. 

X.    Designed  a  methodology  by  which  future  performance 
may  be  evaluated  in  relation  to  the  performance 
specified  in  the  plan 

Interview  Question:    What  do  you  think  you  will 
obtain  (have  obtained)  from  the  AMA's 
training  program? 

Range  of  Scale  Possibilities:     (1)  no  value 
to  (7)  maximum  value. 


Points  of  Time:    Tl^  T2,  T3,  T4. 
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States:    El  £•  E2  only. 
Specific  Methods  of  Evaluation: 

2.  Informal  Feedback 

33 

3.  Performance  Reviews 

4.  Que  s  t i  onn  ai  re  s 

34 

5.  Task  Completion  Inventories 

35 

6.  Unobtrusive  Measures 

Interview  Question:    ^^That  specific  methods  do 
you  use  to  determine  if  the  continuing 
and  specific  objectives  of  your  division 
are  being  mot? 

Range  of  Scale  Possibilities:     (1)  minimum  use 
to  (7)  maximum  use. 

Points  oi  Time:  T4. 

States:     El,  E2,  Cl. 

Tho  Questionnaire  items  are: 


Performance  reviews:    oral  assessment  of  progress 
toward  objectives  by  those  within  the  organization  responsible 
for  thc^ir  attainment.    A  performance  review  can  be  based  on 
many  different  kinds  of  evidence;  from  empirical  data 
gathered  by  a  standardized  testing  program  to  the  judgment 
of  the  person  making  the  review. 

Task  completion  inventories;     a  superficial  assess- 
ment of  whether  tasks  demanded  by  an  objective  have  been 
accomplished,  e.g.,  books  ordered,  money  appropriated, 
teachers  trained/  etc.    Such  an  inventory  is  heavily  quanti- 
tative in  nature  with  little  systematic  attention  paid  to 
the  quality  of  accomplishment. 

35 

Unobtrusive  measures;    indices  which  do  not  neces- 
sarily require  the  knowledge  or  cooperation  of  the  person 
whose  progrcim(s)  are  being  evaluated.    These  measures 
avoid  the  problem  of  reactive  effects,  i.e.,  the  extent 
to  which  the  respondent  tailors  his  answer  to  fit  what  he 
thinks  the  evaiuator  wants  to  hear.    Examples  of  this  index 
include  organizational  documents  and  obrervations. 
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LsuUs^are  Jcnowing  how  good  our 

Points  of  Time:    Tl,  T2,  T3,  T4. 

rL°^^T?^^!^^°"  has  reliable  ways  for  knowing 
now  well  it  is  attaining  its  ob^e'^H^  

Points  of  Time:    T3,  T4. 

L.^^^"^"'^^^^  ob^iectivGs  developed  during 
atiA  training  are  clearly  stated  with  rr^anpr-^ 
to  results  oxpectedl   ^  

Points  of  Time:    T3,  T4. 
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Designed  a  methodology  by  which  future 
performance  may  be  evaluated  in  relation  to 
Item  _1  :  the  performance  specified  in  the  plan  


Fall. 

1970 

Spring.  1971 

Fall. 

1971 

Spring, 

1972 

^2 

Ej  a 

'  S 

^1  ' 

\  E2 

N 
4 

N 
7 

N 
8 

N 

7  1 

N 
5 

N 
3 

N 
6 

N 
5 

Kruska. 

L-Wallis  One-Way  Analysis  of.  Variance 

H=  3. 

5714 

H=2. 

815 

H= 0.0889 

H=  1.200 

SiE.= 

:  NS 

Si2.= 

:  NS 

SiB.= 

:  NS 

Sie.= 

=  NS 

Binomial  Test  c 

)f  Proportions 

P=0. 

144 

P=0. 

436 

P=0. 190 

P=0.' 

426 

SiB.= 

:  NS 

.  SiB.= 

:  NS 

Si2.= 

=  NS 

Sin.: 

=  NS 

Fall  1970 

to 

Sorine  1972 

^1 

E2  5 

E 

2 

N 
4 

N 
6 

N 
7 

N 
5 

Kruska 

1-Wallis  One-Way  Analysis  of  Variance 

H=  6. 

545 

H=  8 

.076 

.02 

SiB.= 

.m 

Binomial  Test 

of  Proportions 

P=  0.008 

P=  0. 

409 

Sig.= 

.01 

Sig.= 

NS 
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Evaluation  Techniques  Used:  Evaluation  Techniques  Used: 

Item_2_   Informal  feedback.  Item  3    Performance  reviews. 


prine.  1972 

Si 

orinp.  1972 

El  § 

5  E2 

El    §  Ci 

El    §  E2 

N 
2 

N 
4 

N 
5 

N 
4 

N 
2 

bin  114. 

N 
5 

|fay  An£ 

N 

5 

N 

2 

N 
2 

N 
2 

N 
5 

N 
2 

H=1.928 
Sig.=  NS 

H=0.015 
Sig.=  NS 

H=2.400 
Sig.=  NS 

ivsis  or  ygi 
H=0.037 

Sig.=  NS 

piance 
H=0.600 

Sig.=  NS 

H=0.150 
Sig.=  NS 

P=  0. 

612 

P=  0.548 

P=  0. 

178 

Sig.= 

NS 

-Sig.=  NS 

Sig.= 

NS 

Binomial  T^gi-  of  Proportion^ 


P=0.002 

P=0.764 

P=0.050 

Sig.=.01 

Sig.=  NS 

Sig.=  .05 

Evaluation  Techniques 
Itein_4      Used:  Questionnaires, 


Krus 


Ej    §  Cj 


N 
3 


N 
4 


E2    §  Cj 


N 
1 


N 
4 


El  §  E2 


N 
3 


N 
1 


tal-Wallis  One-Wav  Analysis  of  Varianrft 


H=  0.281 
Sig.=  NS 


P=  0.850 
Sig.=  NS 


H=  2.000 
Sig.=  NS 


H=  1.  800 
Sig.=:  NS 


Binomial  Test  of  Proportions 


P=  0.146 
Sig.=  NS 


P=  0.104 
Sig.=  NS 


ERIC 
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Evaluation  Techniques  Used:  Evaluation  Techniques 

Item  5^    Task  Completion  InventoriesJtem  6      Used:    Unobtrusive  measures. 


s 

El  5 

Cl 

^2 

^1 

S  E2 

N 
4 

N 
3 

N 
1 

N 
3 

N 
4 

N 
1 

H=4.500 

H=0.200 

H=2.000 

Sig.=  .05 

Sig.=  NS 

Sig.=  NS 

Si 

jrine.  1972 

El    §  Ci 

E2    §  Cj 

El  a  E2 

N 
1 

N 
2 

N 
3 

N 
2 

N 
1 

N 
3 

ilvsis  of  Vai 

fiance 

H=0.000 
Sig.=  N.S  • 

H=0.000 
'Sig.=  NS 

H=0.800 
Sig.J^S 

P=  0.402 
Sig.=  NS 


P=  0.292 
Sig.=  NS 


Binomial  Test  of  Proportions 


P=  0.025 
Sig.=  .01 


P=0.984 

P=0.624 

P=0.292 

Sig.=  NS 

Sig.=  NS 

Sig.=  NS 
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ERIC 
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Item      8        My  organization  has  reliable  wavs 
for  knowing  how  well  it  is  attain- 
ing  its  objectives. 


Fall,  1971 

Spring,  1972 

Experimental  SED«1 

N 
68 

X 

4.220 

SD 

1.563 

N 

X 

4.076 

SD 

1.305 

Experioental  SED*2 

45 

3. $00 

;.232 

40 

3.800 

1.505 

Contxol  SED  «1 

$7 

4.552 

1.282 

61 

4.393 

r.440 

Control  SED  #2 

$1 

4.409 

1.464 

66 

3.969 

1.53^ 

Total 

241 

206 

Two  Way  Analysis  of 
Variance 

F 

Simif. 

Experimental  SED  #1 

^ol. 

0.638 

NS 

ir/Control  SED  #1 

Rpw 

2.933 

NS 

Experimental  SED  #1 

Col. 

2.304 

NS 

w/Control  SED  *  2 

Row 

0.045 

NS 

Experimental  SED  *2 

Col. 

0.011 

NS 

w/Control  SED  *1 

Row 

16. 371 

.001 

Enierimental  SED  #2 

Col. 

0.380 

NS 

w/Control  SED  «2  plow 

6.335 

.0^ 
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Item     9         I  think  that  the  objectives 

developed  during  AMA  training 
are  clearly  stated  v;ith  respect 
to  results  expected. 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  SED#1 
Experimental  SED#2 
Total 

N 

X 

4.  fl^ft 

SD 

N 

39 

X 

5.205 

SD 
1.128 

45 

4,^-77 

1.466 

39 

3.948 

1.571 

78 

Two  Way  Analysis  of 
*'ar  lance 

T3  ST^ 

Experimental  SED#1 
W/Experimental  SED#2 

F 

Sisnif . 

n.nqn 

NS 

17 

nm 
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Analysis  of  Item  #1  indicates  that  at  no  time  between 
the  pre-training  question  and  the  Spring  of  1972  did  any 
change  appear  in  the  degree  of  emphasis  or  amount  of  aware- 
ness between  the  two  Experimental  States. 

However r  over  time  within  the  states  thore  v/ore 
significant  changes.  The  development  of  an  evaluation 
methodology  vras  not  seen  as  a  result  of  AMA  training  in  the 
Spring  of  1972  to  the  extent  that  staff  had  hoped  this 
would  happen r  prior  to  training.  In  both  states,  this  was 
clearly  a  negative  affect  of  training;  the  managers  in 
question  did  not  receive  v/hat  they  expected. 

Items  #2  through  #6  were  attempts  to  measure  the 
extent  to  which  certain  evaluation  techniques  were  used  in 
the  Experimental  and  Control  States.    This  question  was  asked 
in  Spring  1972  only. 

Items  #2,  #4,  and  #6  indicate  no  difference  at  all 
between  the  states  regarding  the  use  of  informal  feedback, 
unobtrusive  measures,  and  questionnaires  as  evaluation 
methods. 

Items  :'f3  and  #5  did  indicate  some  significant  differ- 
ences between  the  states.     In  both  cases  State  El  uses 
more  performance  reviews  and  task  completion  inventories 
as  evaluative  methods  than  either  the  other  Experimental 
State  or  the  Control  State. 

It  should  be  noted  that  possible  responses  to  this 
question  included  seven  other  evaluation  techniques  which 
were  not  mentioned  by  the  respondents  at  all,  or  by  only 
one  or  two  persons.    This  fact,  taken  with  the  IV  s  reported 
above,  suggests  somev/hat  inadequate  knowledge  and  use  of 
various  evaluative  techniques  by  top  managers  in  these  states. 


Item  #7!    Fall  1970  to  Spring  1971 

Neither  El  nor  E2  exhibited  any  effects  of  train- 
ing during  this  period.    E2  did  not  feel  so  positive  as  CI 
about  its  ways  for  knowing  how  good  its  results  are^  but 
this  difference  v;as  as  great  after  training  as  it  had  been 
before.    No  differences  existed  between  El  and  CI  nor  did 
El  change  its  perceptions  over  time. 


Item  ^il:    Spring  1971  to  Fall  1971 

Analysis  revealed  no  significant  changes  in  El  or 
E2;  they  did  not  consider  their  evaluation  methods  any 
better  or  any  worse.    CI  remained  the  same  as  El  but 
increased  the  extent  to  which  means  of  assessing  results 
were  thought  superior  to  the  means  used  in  E2. 
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Item  frl:    Fall  1971  to  Spring  1972 

Differences  between  E2  and  CI  in  how  they  felt  about 
the  methods  used  to  assess  results  are  narrowed  between  Fall 
1971  and  Spring  1972,  but  remained  significant.    When  com- 
pared to  C2,  E2  showed  a  similar  perception  of  this  variable. 
Looking  at  El  and  C1/C2,  we  can  determine  no  measurable 
differences  between  them.    There  were  no  significant  changes 
in  either  Experimental  State  over  time. 


Item  jt7;    Fall  1970  to  Spring  1972 

An  assessment  based  on  a  compa-ison  of  data  gathered 
before  and  after  training  (Fall  1970  and  Spring  1972 
respectively)  must  conclude  that  AMA  training  had  negative 
effects  in  El  and  positive  effects  in  \2.    Although  a  pre- 
and  post-training  analyse '  reveals  net  significant  declines 
in  E2  and  Cl's  opinion  of  their  evaluation  methods,  CI 
declined  at  a  faster  rate  than  E2  and  a  great  difference 
appeared  between  them.    El  reported  decreases  on  this  item 
which  were  greater  than  CI  and  falling  at  a  more  rapid  rate. 
Differences  between  El  and  CI  were  not,  however,  at  signifi- 
cant levels.    Nevertheless,  negative  effects  of  AMA  training 
in  El  on  this  varicible  must  be  registered. 


Item  #8;    Fall  1971  to  Spring  1972 

Item  #8,  My  organization  has  reliable  ways  for  knowing 
how  well  it  is  achieving  its  objectives ,„  is  related  to  Fall 
X971  and  Spring  1972  only.    While  El  and  E2  did  not  change 
their  opinions  between  T3  and  T4,  both  generally  considered 
their  methods  for  assessing  whether  or  not  they  were 
accomplishing  their  objectives  to  be  less  reliable  than  did 
either  Control  State.    El,  however,  reversed  its  position 
relative  to  C2  in  the  Spring.    In  the  case  of  F.2,  no 
reversal  occurred  and  its  differences  with  CI  and  C2  remained 
significant. 


Item  ft9:    Fall  1970  to  Spring  1972 

Data  on  how  persons  think  that  the  objectives  developed 
during  AMA  training  are  clearly  stated  with  respect  results 
expected  could  only  be  gathered  from  the  tv;o  Experimental 
States  during  Fall  1971  and  Spring  1972.    The  results  indi- 
cated that  there  is  a  significant  difference  between  El  and 
E2;  E2  does  not  feel  that  its  objectives  are  as  clearly 
stated  as  does  El.    Ho  major  changes  over  time  for  either 
Experimental  State  can  be  reported.    However,  the  absence 
of  a  Control  State  urges  caution  in  ascribing  any  training 
effects  to  any  State. 
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DATA  SUriMARY 
Development  of 
Evaluation  Techniques 

Fall,  1970-Spring,  1972 
Type  of  Data 


IMPACT  OF  TRAINING 


Item 


CONTENT 


Positive         No  Negative 
Effect       Effect  Effect 


1 

Designed  a  methodology  by 
vdiich  future  performance 
may  be  evaluated  in  rela- 
tion to  the  performance 
specified  in  the  plan 

ElrE2 

2 

Informal  Feedback 

Insufficient 

3 

Performance  Reviews 

data  (T4  only) 

4 

Questionnaires 

on  which  to 

5 

Task  Completion  Inven-- 
tories 

base  a 

6 

Unobtrusive-  ricasurGS 
QUESTIONNAIRE 

judgment. 

7 

I  have  good  ways  for 
knowing  how  good  our 
results  are. 

E2 

El 

8 

•iy  organization  has  re- 
liable ways  for  knaving 
hovf  v;ell  it  is  attaining 
its  objectives. 

E1,E2 

9 

I  think  that  the  objec- 
tives developed  during 
AMA  training  are  clearly 
stated  with  respect  to 
results  expected. 

E1,E2 

Evaluation  is  an  underdevelooed  area  of  the  planning 
process  in  both  Experimental  States.    Of  those  items  for 
vrhich  effects  could  be  determined  ^  there  were  many  more 
negative  than  positive  effects  of  training. 


The  development  of  an  evaluation  system  (Item  #1)  was 
thought  more  important  in  both  States  prior  to  training  than 


18  f 


it  was  after  training.    This  suggests  that  trainee  expecta- 
tions were  not  realized;  the  Experimental  States  anticipated 
more  than  the  AMA  program  was  able  to  deliver  in  this  area. 

The  questionnaire  item  about  means  of  assessing 
results  (Item  #7)  reported  that  as  a  result  of  training  El 
considered  its  ways  worse  off  while  E2  thought  they  had 
benefited  from  training.    This  difference  in  assigned 
effects  v;as  the  result  of  both  States  declining  in  their 
opinions  of  their  own  evaluation  techniques.    El  declined 
faster  than  the  Control  State,  which  constituted  a  negative 
effect,  while  E2  declined  more  slowly  than  the  Control, 
which  won  its  positive  rating.    Attributing  oositive  train- 
ing effects  on  tiic  basis  of  a  slower  decline"  should  not 
particularly  encourago  those  who  are  concerned  with  educa- 
tional evaluation. 

The  remaining  questionnaire  items  revealed  that 
training  had  no  effect;  however,  contradictory  changes  did 
occur.    The  agencies'  evaluative  techniques  measured  by  ' 
Item       became  less  reliable  in  El  and  more  reliable  in  F2. 
Looking  at  attitudes  expressed  in  Item  9  about  how  ooera- 
tional  v/ere  the  goals  developed  during  ATlA  training,' wc 
found  that  El  thought  theirs  were  more  operational  over 
time  while  E2  believed  theirs  v;ere  less  so. 

The  blank  spaces  on  the  summciry  sheet  refer  to  the 
inadequate  nature  of  the  data  gathered  for  these  items, 
corrected  in  T4  only.    The  AMA  cannot  be  held  accountable 
for  any  of  those  evaluative  variables  on  the  basis  of  such 
limited  information. 

We  have  already  pointed  out  that,  aside  from  train- 
ing effects,  the  six  alternative  evaluative  techniques  listed 
in  the  data  summary  are  those  cited  by  more  than  two  persons 
when  asked  an  open-ended  question  about  what  techniques  they 
used.    Performance  reviowa  were  mentioned  most  often  in 
El;  E2  favored  the  informal  feedback  approach  to  evaluation. 
The  traditional  educational  measures  such  as  custom-made 
and  standardized  tests,  written  progress  reports,  and  outside 
evaluators,  were  hardly  mentioned  at  all.    This  lack  of 
multiple  indices,  the  basis  of  any  reliable  evaluative 
system,  suggests  that  both  States  have  much  work  to  do  in 
this  crucial  activity. 

For  these  reasons,  the  AMA  program  had  mixed  effects 
on  the  development  of  evaluative  techniques.    One  item  in 
one  State  is  positive  while  the  other  seven  items  in  both 
States  were  negative  or  showed  no  effects  at  all.  Consider- 
ing the  importance  of  evaluation  to  education,  the  inability 
of  the  Mik  program  to  help  the  states  improve  performance 
m  this  area  is  unfortunate. 
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B.    Top  Management  Support  for  Planning 

Inherent  in  the  AIlA's  training  design  was  the  absolute 
necessity  that  top  inanagemGnt  scpport  the  planning  process. 
The  following  eight  items  were  designed  to  obtain  perceptions 
of  organization  members  about  that  support  (or  its  absence) 
and  to  record  any  change  over  time. 

1.  Adequate  Resources  (money  and  information) 

2.  Control  System  expressed  throucyh  decisionmaking 
process 

Interview  Question:    What  are  the  roadblocks 
to  change  in  this  organization? 

Range  of  Scala  Possibilities;     (1)  major  road- 
block/always stops  change  to  (7)  weak 
rofidb lock/seldom  stops  desired  change. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:     El,  E2,  CI. 

The  questionnaire  items  are: 

3.  Ily  manager  makes  it  clear  that  he  is  committed 
to  the  success  of  our  projects. 

Points  of  Timr:     Tl,  T2,  T3,  T4. 

4.  My  manager  has  expressed  the  belief  that  the 
xWJs  training  program  has  been  hGlpful. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:    This  item  was  asked  in  the  Experimental 
States  only. 

5.  !iv  manager  understands  planning  theory  and  is 
able  to  put  it  into  practice. 

Points  of  Time:    T3,  T4. 

6.  I  believe  my  organization  gives  me  adequate 
training  to  do  my  work  effectively. 

Points  of  Time:    T3,  T4. 

7.  I  feel  good  about  my  manager's  ability  to  plan. 
Points  of  Time:    T3,  T4. 
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My  manager  provides  roe  with  adequate  support 
to  perform  my  job.  ~~ 

Points  of  Time:    T3/  T4. 
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Item       5       My  manager  understands  planning 

theory  and  .is  able  to  put  it  into 
practice. 


Fall,  1971 
T 

Spring,  1972 

Experimental  SED#1 
Experimental  SED#2 
Control  SED  *1 
Control  SED  #2 
Total 

N 

68 

X 

4.897 

SD 

1.702 

N 

39 

X 

5.025 

SD 
1.404 

45 

4'.  511 

1.60^ 

40 

4.025 

1.576 

T7 

■■  4.540 

1.  48€ 

61 

4.721 

1.345 

T.r 

4.590 

1.79: 

66 

4.530 

1.303 

241 
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TWO  Wsy  Analysis  of 
Variance 

Experimental  SED  #1 
W/Control  SED  «1 
E]qperimental  SED  #1 
w/Control  SED  f  2 
Ejqperimental  SED  #2 
w/Control  SED  #1 
Experimental  SED  #2 
w/Control  SED  #2 

F 

Sianif. 

|0l. 

0.050 

NS 

Row 

0.419 

NS 

Col. 

0.026 

NS 

tow 

3.596 

NS 

Col. 

2.  849 

NS 

Row 

.01 

Col. 

1.533 

NS 

tow 

1.756 

■    MS  " 
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I  believe  mv  oraaniza1;:i9n  aiveg  j^f^ 
adequate  training  to  do  mv  work 
effectively. 


F 

•all,  1971 

Spring,  1972 

Experimental  SED#1 
Experinental  SED#2 
Control  SBD  *1 
Control  SED  ff2 
Total 

N 

68 

X 

5. 102 

SD 

N 

...40. 

X 

SD 

.1.264 

45 

4.844 

,-i.4?? 

49 

4.550 

1.395 

67 

4.880 

1.571 

61 

4.852 

1.424 

4.754 

1.608 

66 

4.651 

1.341 

HT 

207 

1.  

Two  Analysis  of 
Variance 


Experimental  SED  #1 
W/Control  SED  *1 
Experimental  .SED  »1 
w/Control  SED  f  2 
Experimental  SED  #2 
w/Control  SED  #1 
Experimental  SED  #2 
w/Control  SED  #2 


:ol, 


Row 


Col. 


low 


Col, 


low 


:oi. 


low 


0.000 


1.570 


0.042 


4.411 


0.614 
0.677 


0.950 
0.000 


NS 


NS 


NS 


,05 


NS 


"nS" 


NS 
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Item 


I  feel  cjood  about  mv  manager's 
ability  to  plan.  


Fall,  1971 
^3 

Spring,  1972 

Experimental  SEDil 
Experinental  SED#2 
Control  SED  *1 
Control  SED  «2 
Total 

N 

6i 

X 

5'.  161 

SD 

1.653 

N 

40 

X 

5.525 

SD 
1.131 

4£ 

4:733 

1.586 

■  ^9 

4.  230 

1.580 

67 

5.447 

1.222 

61 

5.245 

1.337 

6] 

5.131 

1.^17 

5.090 

1.249 

243 
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Two  Way  Analysis  of 
Variance 


Experimental  SED  #1 
W/Control  SED  «1 
Ejqperimental  SED  «1 
w/Control  SED  *  2 
Esqperimental  SED  #2 
w/Control  SED  #1 
Experimental  SED  #2 
w/Control  SED  #2 


F 

Sienif. 

col. 

0.193 

NS 

Row 

0.000 

NS 

Col. 

0.688 

NS 

Row 

1.424 

NS 

Col. 

3.166 

NS 

Row 

19.091 

.001 

Col. 

1.654 

NS 

|Row 

8.887 

.61 
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Item       3      My  manager  providps  ttip  with  adP>qiiai-P 
support  to  .perform  mv  inh..  


Fall,  1971 
T 

Spring,  1972 

Experimental  SED91 
E3q>eriBental  SBD#2 
Control  SED  »1 
Control  SED  #2 
Total 

 1 

N 

68 

X 

5.691 

SD 
L.187 

N 
-  .40 

X 

-5.475 

SD 
1.240 

45 

5.044 

L.364 

3? 

4.410 

1.584 

67 

5.641 

L.214 

60 

5.183 

1.455 

61 

5.^5^^ 

L.565 

66 

5.257 

1. 180 

ur 
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TWO  Way  Analysis  of 
Variance 


F 

Simif. 

Experiaental  SED  #1 

f;oi. 

3. 

912 

W/Control  SED  «1 

Row 

0. 

999 

NS 

Experiaental  SED  #1 

Col. 

2. 

201 

NS 

w/Control  SED  «  2 

Row 

1. 

019 

NS 

Experiaental  SED  #2 

Col. 

7: 

•761 

.01 

w/Control  SED  #1 

Row 

12. 

T40 

.001 

Experimental  SED  #2 

Col. 

6. 

m 

.05 

w/Control  SED  #2 

low 

11. 

621 

.001 
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Item  iil,  Adequate  Resources  (money  and  infonnatlon) 
reform  ca  a  pcrcreived  obstacle  to  organizational  change. 
Analysis  of  the  data  indicates  that  inadequate  resources 
hindered  change  as  greatly  after  the  training  as  they  did 
before  it.    No  differences  existed  in  emphasis  or  awareness 
between  the  Experimental  States,  nor  did  they  change  over 
time.    The  AIIA  training  had  no  effect  on  this  variable  as 
a  roadblock  to  change. 

Item  ^r2,  Control  System  expressed  through  decision- 
making  process ,  concerns  another  possible  hindrance.  Com- 
parisons of  the  degree  of  eit?>hasis  placed  upon  the  Control 
system  as  a  roadblock  to  change  reveals  no  differences 
between  the  States  before  or  after  training.    Neither  El, 
E2  or  CI  changed  their  impressions  over  time. 

In  terms  of  awareness  of  this  issue,  El  and  E2  were 
more  perceptive  than  CI  beforo  training.    El  was  also  more 
aware  than  CI  in  Fall  1971.    Examining  changes  over  time 
(Fall  1970  to  Spring  1972),  El  and  E2  grew  less  aware  of 
their  Control  systems  as  an  obstacle  after  training  than 
they  had  been  beforo  they  entc^rod  training. 


Item  #3;     Fall  1970  to  Spring  1971 

During  the  immediate  pro-  post- training  period.  El 
reflected  significant  change  in  the  extent  to  which 
manager  makes  it  clear  he  is  committed  to  the  success ^of 
our  projects;  this  can  be  attributed  to  training.    In  El 
the  effect  was  to  increase  managers'  commitment.    In  organi 
zation  E2  the  frequency  with  which  managers  expressed  com- 
mitment to  the  success  of  projects  was  unchanged.    Both  of 
the  Experimental  States  differed  (independent  of  time)  from 
the  Control  on  the  grounds  of  how  great  a  commitment  was 
expressed  by  managers.    Managers  in  El  expressed  signifi- 
cantly more  commitment  than  the  Control  State's  managers 
and  managers  in  E2  expressed  significantly  less  commitment. 


Item  #3;    Spring  1971  to  Fall  1971 

The  pattern  established  in  T1-T2  v/as  reversed  in  thi 
period.    Instead  of  further  increases  in  managerial  commit- 
ment. El  showed  a  decrease,  albeit  insignificant.  This 
decrease  did  not  change  the  significant  difference  between 
El  and  CI;  this  Experimental  State  continued  to  feel  their 
managers  were  more  committed  than  those  in  CI.    On  the 
other  hand,  there  were  negative  effects  in  E2  which  were 
the  result  of  training.    Although  managers  in  E2  had  become 
more  involved  with  their  subordinates'  projects  in  the  Fall 
as  compared  to  the  previous  Spring,  their  counterparts  in 
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CI  had  also  grown  more  involvod— only  more  so.    Since  thti 
Control  state,  without  training,  had  increased  faster  than 
the  Experimental  State,  with  training,  negative  training 
effects  occurred  in  E2. 


Item  f>3t    Fall  1971  to  Spring  1972 

Between  the  Fall  of  1971  and  the  Spring  of  1972,  El 
continued  to  report  that  their  managers  were  significantly 
more  committed  to  project  success  thsm  did  CI  or  C2.  But 
this  Experimental  State  did  not  really  change  during  this 
time.    Thus  wo  cemnot  conclude  that  any  effects,  positive 
or  negative,  occurred  in  El  because  they  had  been  trained 
in  planning  techniques  by  ATIA.    In  comparison  to  CI,  E2 
developed  even  more  statistical      difference  than  during 
T2-T3;  this  difference  (accompanied  by  a  definite  decline 
over  time)  constitutes  another  negative  effect  of  training. 
Managers  in  E2  were  also  viewed  less  favorably  in  S2  than 
in  C2;  yet  an  E2/C1  comparison  indicates  no  major  changes 
between  T3-T4  and  therefore  no  training  effect. 


Item  #3;    Fall  1970  to  Spring  1972 

An  analysis  of  both  Experimental  States  between  Fall 
1970  and  Spring  1972  reveals  that  training  affected  each 
organization  differently.    El  underv;ent  a  decline  in  the 
degree  to  which  managers  felt  strongly  about  what  their 
people  achieved;  but  the  decline  in  CI  over  the  same  period 
was  even  more  pronounced,  creating  clear  differences 
between  them.    Hence,  AMA  training  stabilized  El  in  relation 
to  CI.    The  way  persons  in  E2  viewed  managerial  commitment 
changed  considerably  over  time;  they  thought  their  managers 
were  much  less  committed  one  year  after  training  than  they 
had  been  before.    Since  the  decline  in  P2  corresponded  to  a 
similar,  but  slower,  decline  in  CI,  and  was  accompanied  by 
greater  E2  instability,  AMA  training  influenced  managers  in 
this  Experimental  State  to  be  less  clear  about  their  commit- 
ment to  the  success  of  organizational  projects. 


Item  i^4;    Fall  1970  to  Spring  1971 

The  fourth  item.  My  manager  has  expressed  the  belief 
that  the  AMA's  training  program  has  been  helpful,  was  not 
administered  in  the  questionnaire  given  to  the  Control  State. 
Therefore,  interpretation  of  this  item  is  somewhat  less 
reliable.    The  tinalysis  is  limited  to  a  comparison  of  the 
two  Experimental  States  during  the  Fall  of  1970  to  Spring 
1971.    Analysis  indicated  that  the  Experimental  States 
differed  in  both  the  extent  to  which  positive  attitudes 
toward  the  AMA  wore  held  by  managers  and  the  amount  of 
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emphasis  thoy  gave  their  attitude.    The  first  difference 
was  due  to  a  significantly  greater  amount  of  belief  being 
expressed  by  the  managers  of  El;  the  second  difference  vjas 
due  to  E2's  managers  increasing  the  frequency  v/ith  which 
they  expressed  positive  cittitudes  toward  the  training  that 
had  been  received.    Since  E2  remained  unchanged  and  FX  did 
not,  we  interpret  this  to  mean  that  a  training  effect 
existed  in  El  and  not  in  E2. 


Item  #4;     Spring  1971  to  Fall  1971 

Neither  Experimental  State  changed  between  Spring 
and  Fall  1971.    Managers  in  E2  were  not  as  expressive  as 
those  in  El  eibout  the  benefits  of  ATIA    training;  this 
difference  betv/een  the  States  is  significant.    No  effect  of 
training,  however,  can  be  assumed  due  to  the  absence  of 
change  over  time* 


Item  »4t    Fall  1971  to  Spring  1972 

There  were  no  effects  attributable  to  A?1A  training 
during  this  time.  Massive  differences  continued  to  exist 
between  El  and  E2  along  the  same  lines  as  previously;  but 
the  greater  frequency  with  which  positive  opinions  were 
expressed  by  E2  managers  as  compared  to  El  managers  vms 
still  insufficient  to  produce  importemt  difference  within 
the  States  over  time. 


Item  #4;    Fall  1970  to  Spring  1972 

The  pattern  observed  during  T2-T3  and  T3-T4  also 
characterized  the  overall  assessment.    Managers  in  F.2 
wore  considered  less  favorably  disposed  to  the  A!1A  program; 
howovor,  the  relative  stability  of  this  difference  pre-  and 
post-training  meant  that  training  did  not  affect  this  variabl 
in  El  or  E2. 


Item  ?^5:    Fall  1971  to  Spring  1972 

Item  5,  lly  manager  understands  plannini^  theory  and 
is  able  to  put  it  into  practice,  is  another  item  relevant 
to  the  Experimental  States  as  well  as  two  Control  States 
only  at  two  points  of  time:    Fall  1971  and  Spring  1972. 
Because  we  have  no  baseline  data  on  this  item  gathered  before 
training,  we  must  be  cautious  about  attributing  any  changes 
or  differences  to  training.    This  caution  is  unnecessary 
here,  because  no  significant  changes  or  differences  occurred 
in  or  between  the  States.    Thc>  only  exception  to  this  general 
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rule  is  that  E2  seems  to  feci  that  its  managers  are  less 
competent  in  planning  than  thoso  in  CI.    This  being  the 
only  difference,  it  must  be  concluded  that  in  El  and  E2, 
managerial  understanding  of  planning  cheery  and  practice 
as  well  an  managerial  ability  to  put  it  into  practice  was 
unaffectetl  by  training. 


Item  #6;    Fall  1971  to  Spring  1972 

I  believe  my  organization  gives  me  adequate  training 
to  do  my" work  effectively.  Item  6,  shows  no  differences, 
with  the  exception  of  E1/C2,  as  we  review  what  these 
respondents  feel  about  their  in-service  training  programs. 
If  we  examine  the  exception,  we  find  that  C2  does  not  feel 
as  positive  as  El  about  the  training  they  have  received. 
No  reportable  changes  exist  within  any  State  between  Pall 
1971  and  Spring  1972. 


Item  #7;    Fall  1971  to  Spring  1972 

When  evaluating  over  time  El  and  E2  in  terms  of  Item 
v7,  I  feel  good  about  my  mcuiager's  ability  to  plan,  an 
interesting  pattern  emerges.    Between  T3-T4,  El  did  not 
increase  their  respect  for  their  managers'  planning  skills, 
nor  were  they  significantly  different  in  this  area  from  CI 
or  C2.    However,  E2,  a  State  that  wont  through  P^^IA  training, 
felt  worse  about  their  management's  planning  competence  than 
the  Control  States,  which  had  received  no  comparable  training. 
This  difference  was  not  the  result  of  any  State's  changing 
over  time  but  of  the  fact  that  E2  in  T3  scored  lower  than 
CI  or  C2  and  remained  in  T4  in  the  same  relative  position. 


Item  ft8;    Fall  1971  to  Spring  1972 

Item  #8,  My  m3.nager  provides  me  with  adequate  support 
to  perform  my  job,  was  applied  in  T3  and  T4  to  all  States. 
In  both  periods,  E2  reported  that  the  support  they  were  able 
to  obtain  in  performing  their  jobs  was  not  as  adequate  as 
the  aid  provided  in  CI  or  C2.    While  both  Control  States 
had  declined  over  this  period,  E2  had  declined  even  more 
sharply.    Making  the  transition  to  El,  we  find  it  differs 
little  from  CI  or  C2  but,  compared  to  CI,  El  has  experienced 
a  major  decline  in  perceived  support.    However,  if  El  fell 
off  a  little,  CI  fell  off  steeply.    Since  we  have  no 
indication  of  how  the  Experimental  States  felt  about  the 
work-related  environment  before  they  entered  AMA  training, 
wo  must  be  careful  about  presuming  that  change  was  caused 
by  AJ1A  intervention. 
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DATA  SUItli-iRY 


Top  Management  Sup- 
port for  Planning 

Fall.  1970-Spring,  1972  ^^^^^^  TRAINING 


Item 

Type  of  Data 
CONTENT 

Positive 
Effect 

NO 

Effect 

Negative 
Effect 

1 

Adequate  Resources  (money 
and  information) 

E1,E2 

2 

Control  Systom  expressed 
through  decisionmaking 
process 

QUESTIONNAIRE 

E1,E2 

3 

Hy  manager  makes  it  clear  he 
is  committed  to  the  success 
of  our  projects. 

El 

E2 

4 

My  manager  has  expressed  the 
belief  that  the  AMA's  train- 
ing program  has  been  helpful. 

E1,E2 

5 

My  manager  understands  plan- 
ning theory  and  is  able  to 
put  it  into  practice. 

El,E2 

6 

I  believe  my  organization 
gives  me  adequate  training 
to  do  my  work  effectively. 

E1,E2 

7 

I  feel  good  about  my  manager* 
ability  to  plan. 

S 

E1,E2 

8 

fly  manager  provides  me  with 
adequate  support  to  perfoxrm 
my  job. 

El 

E2 

Based  on  the  Content  items,  we  found  that  in  the 
Experimental  States  the  training  program  had  no  effect  on 
attitudes  of  top  administrators  toward  inadequate  resources 
and  the  decisionmaking  process  as  obstacles  to  organiza- 
tional change.    The  States  did  not  alter  their  emphases  on 
the  lack  of  money  and  adequate  inforration— and  the  presence 


2C^ 


of  Cofitrols  inherent  in  the  decisionmaking  process--as 
barriers  which  kept  them  from  moving  forward.    One  slight 
shift  appeared  in  their  awareness  of  these  variables  as 
roadblocks;  both  States  mentioned  them  less  in  the  Spring 
of  1972  than  they  had  before  training  began  in  the  Fall  of 
1970. 

Two  questionnaire  items  made  it  possible  to  determine 
that  the  ATIA  program  affected  the  Experimental  States.  Here^ 
the  States  wore  moving  in  opposite  directions.  Attitudes 
about  the  degree  of  commitment  expressed  about  projects 
(Item  #3)  and  about  support  provided  for  project  accomplish- 
ment (Item  #8)  were  partially  affected  in  El  and  negatively 
influenced  in  E2. 

The  positive  effects  attributed  to  El  and  negative 
effects  assigned  to  E2  were  the  only  training  effects  shewn 
on  top  management  support  for  planning.    Aside  from  the 
revelation  of  these  two  questionnaire  itemSf  the  analysis 
brought  out  no  other  effects. 

Interpretations  of  the  questionnaires  must  be 
moderated  because  of  the  absence  of  pre-training  data  on 
most  items.    Without  knowing  how  the  organizations  felt 
before  training^  we  have  no  baseline  data  against  which 
to  compare  how  they  feel  now.    We  lack  direct  nvidence, 
thereforo^  on  which  to' build  a  conclusive  argument  for  the 
effects  we  did  or  did  not  identify. 

By  evaluating  the  relative  standing  of  the  Experi- 
mental States  on  these  concerns,  separate  from  a  considera- 
tion of  training  effects,  a  very  stable  ranking  is  revealed. 
In  almost  every  comparison,  El  sensed  the  greatest  managerial 
support  for  planning  during  T3-T4.    Conversely,  the  least 
support  is  perceived  by  respondents  in  E2.    The  two  Control 
States  assume  a  middle  position  between  the  Experimental 
States  on  virtually  every  item. 


C.    The  Credibility  of  the  Planning  Process 

The  following  data  examine    the  assumption  that 
planning,  to  be  effective  within  the  oarganizations--  must  be 
thought  credible.    We  are  interested  in  how  important 
planning  is  to  the  SED's,  and  how  the  training  program  may 
have  redefined  planning* s  role.    There  are  eight  items  in 
this  sub-category,  four  from  the  interview  data  and  four  from 
the  questionnaire. 

1.    Establish  credibility  of  Planning 

Interview  Question;    What  do  you  think  you  will 

obtain  (have  obtained)  from  the  AMA  training? 

ERIC 
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Range  of  Scale  Possibilities:     (1)  no  value 
to  (7)  maximum  value 

Points  of  Tine:    Tl,  T2,  T3,  T4. 

States:    El  &  E2  only. 

2.  Role  of  Planning:    how  integral 

3.  Role  of  Planrdng:    how  much  is  needed 

4.  Role  of  Planning:  emergence 

Interview  Question:    What  is  the  role  of  planning 
in  running  the  state's  schools? 

Range  of  Scale  Possibilities:    how  integral: 
(1)  no  value  to  (7)  integral  part, 
how  much  is  needed:     (1)  no  value/should 
not  be  used  at  all  to  (7)  everything 
should  be  planned. 

emergence:     (1)  still  not  used  to  (7)  long- 
standing practice. 

Points  of  Tine:    Tl,  T2,  T3,  T4. 

States:    El,  E2,  &  CI. 

The  Questionnaire  items  are: 

5.  AS  I  see  it,  planning  is  an  integral  part  of 
running  the  state' o  schools. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

6.  As  I  see  it«  persons  in  this  organization  put  a 
Tot  of  e£tort  into  planning. 

Points  of  Time:    T3,  T4. 

7.  My  capability  to  plan  effectively  will  posi-' 
tively  affect  my  future  career  in  this 
organization. 

Points  of  Tine:    T3,  T4. 

8.  The  activities  relating  to  Planning  aire  having 
arToffect  on  the  policy  of  this  organization. 


Points  of  Time:    T3,  T4. 
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Item     1  :    Establish  credibility  of  Planning 


Fall.  1970 

Spring,  1971 

Fall.  1971 

Spring,  1972 

El  5  E2 

\    5  ^2 

E^      §  E2 

^1      5  ^2 

N 
7 

N 
8 

N 
8 

N 

11 

N  N 
9  10 

N 
9 

N 
8 

Kruska 

L-Wallis  One-Way  Analysis  of  Variance 

H=  1.208 
Sie.=  NS 

H=  0..615 
Si2.=  NS 

H=  1.500 
Sie.=  NS 

H=8.898 
Sig.= . 01 

Binomial  Test  of  Proportions 

P=0.736 
Si2.=  NS 

P=0.026 
Sie.=  NS 

P=1.00 
SiE.=  NS 

P=0.000 
Sig.=  NS 

Fall  1970         to               Spring  1972 

.5  E^ 

E2           §  E^ 

N 

7 

N 
9 

N  N 
8  8 

Kruska. 

.-Wallis  One-Way  Analysis  of  Variance 

H=  0.100 

Qirj  =  NS 

H=  8.040 
Sip.=  .01 

Binomial  Test  ( 

Df  Proportions 

P=  0.000001 
Sig.=  .001 


P=  0.104 
Sig.=  NS 


T 
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Item      6      As  I  see  it^  persons  in  this  organization 
put  a  lot  of  effort  into  planning. 


Fall,  1971 
^3 

Spring,  1972 

Experiaental  SEDtl 

N 

68 

X 

5.147 

SD 

1.260 

N 

40 

X 

4.625 

SD 
1.212 

Experiaental  SED«2 

45 

4:933 

1.355 

40 

4 

.075 

1.366 

Control  SED  *1 

67 

4.  6S6 

1.409 

60 

4 

.550 

1.281 

Control  SED  #2 

4.934 

1.515 

66 

4 

.439 

1.254 

Total 

< 

24: 
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IVro  Vay  Analysis  of 
Variance 

F 

Sisnif. 

Experimental  SED  #1 

fol. 

6.012 

.05 

W/Control  SED  «1 

I^ow 

-     NS  , 

Experimental  SED  #1 

Col. 

8.303 

.01 

w/Control  SED  *  2 

Row 

1.273 

NS 

Ej^eriaental  SED  #2 

Col. 

9.779 

.01 

w/Control  SED  «1 

1.233 

NS 

Ejqperimental  SED  #2 

Col. 

12.289 

.001 

w/Control  SED  «2 

Row 

0.846 

NS 
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Item      7      My  capability  to  piian  effectively 
will  positively  aff/ect  my  future 
career  in  this  organization. 


Fall,  1971 
T 

Spring,  1972 

Experimental  SEDfl 
Experloental  SED#2 
Control  SED  *1 
Control  SED  «2 
Total 

N 

68 

X 

5.764 

SD 

1.35(3 

N 

X 

'i.4fil 

SD 

.i.n9fi 

45 

5.022 

1.815 

4. "164 

67 

5.555 

1.317 

fil 

1.505 

^1 

5.098 

1.660 

.  66 

>41 
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Two  Way  Analysis  of  |  Tj  S  T 

Variance  n 


F 

ISianif. 

Experimental  SED  f 1 

2U 

ir/Control  SED  «1 

Row 

167 

NS 

E3q)eriifiental  SED  #1 

Col, 

060 

NS 

w/Control  SED  «  2 

Rotf 

8, 

796 

.01 

Ejqperimental  SED  #2 

Col, 

5. 

112 

.05 

w/Control  SED  #1 

Row 

5. 

335 

.05 

Ejqperiaental  SED  #2 

Col. 

1. 

460 

NS 

w/Control  SED  92 

Row 

1. 

385 

NS 
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Item  8 


The  activities  relating  to  planning 
are  having  .an  effect  on  the  policy 
ot  tnis  organization. 


Experimental  SEDIl 
Experiaental  SED#2 
Control  SED  #1 
Control  SED  #2 
Total 


Fall,  1971 

Spring,  1972 

T 

N 

X 

SD 

N 

X 

SD 

M 

5.250 

1.713 

40 

1.141 

45 

5.088 

1.411 

'40 

4.250, 

1.424 

67 

4.850 

1.416 

59 

4-694 

1.392 

61 

4.737 

1.504 

64 

4.484 

1.297 

241 
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TWO  Nay  Analysis  of 
Variance 


Experinental  SBD  #1 
W/Control  SED  *1 
Ejqperinental  SED  »1 
w/Control  SED  f  2 
Experimental  SED  #2 
w/Control  SED  #1 
Eiqperimental  SED  #2 
w/Control  SED  ff2 


col. 


Row 


Col, 


Row 


Col, 


tow 


:oi, 


low 


.6.912 


Q-207 


11.943 


5.087 


0.0724 


6.  264 


0.  299 


JiS. 


001 


.05 


NS 


05 


NS 
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Itein#l,  Establish  crodibility  of  planning^  vras 
applied  to  El  and  E2  only.     It  refers  ',0  a  possible  benefit 
which  they  might  obtain  from  AI^'A  training.    There  were  no 
differences  in  the  value  each  ascribed  to  this  variable 
prior  to  training.    No  diffGrencos  existed  in  the  two 
periods  immediately  after  training  (Spring  1971  and  Fall 
i971).     However,  El  reported  that  training  established  the 
greater  crodibility  of  planning  in  their  organization  than 
in  E2  in  Spring  1972.    E2  also  attached  less  emphasis  to 
this  variable  as  an  effect  of  training  in  Spring  1972  than 
they  had  in  Fall  1970.    No  change  occurred  in  Fl. 

Awareness  of  planning  credibility  followed  a  similar 
pattern.    El  changed  over  the  course  of  the  two-year  evalua 
tion;  this  change  brought  increased  awareness.    El's  change 
was  not,  however,  sufficient  to  create  a  significant 
difference  between  it  and  E2,  before  or  after  training. 

Because  there  was  no  Control  State  against  which  to 
compare  Experimental  State  responses  on  this  item,  results 
should  be  modestly  interpreted.  Nevertheless,  we  can 
conclude  that  E2  was  less  affected  by  training  (regarding 
the  establishment  of  planning 's  credibility)  than  they  had 
predicted  earlier.    This  is  a  negative  effect  of  training. 


Due  to  the  addition  of  a  Control  group.  Item  #2, 
Role  of  Planning;    how  integral,  provides  a  more  solid  base 
on  which  to  interpret  effects.    Analysis  demonstrates  no 
differences  in  E1/E2  awareness,  since  all  respondents 
addressed  this  issue. 

Significant  differences  do  appear  when  we  assess^ 
the  impact  of  training  on  the  extent  to  which  planning  is 
seen  as  an  integral  part  of  running  the  state's  schools. 
Although  El  and  Cl  felt  substantially  the  same  before  train 
ing,  a  significant  difference  was  measured  after  training, 
in  Fall  1971  and  Spring  1972.    The  difference  consisted  of 
El's  attaching  greater' importance  to  planning.    The  reverse 
was  true  for  E2  and  Cl.    Compared  to  the  Control  State, ^ 
E2  viewed  planning  as  a  less  integral  part  before  training. 
After  training,  however,  there  was  no  difference  between 
E2  and  Cl.    El  moved  from  no  difference  with  Cl  to  a 
superior  position?  E2  went  from  an  inferior  position*  to  no 
difference.    Both  Experimental  States  improved  their 
position  on  this  variable  when  compared  with  the  Control 
State. 

These  changes  were  due  not  so  much  to  heightened 
awareness  in  El  or  E2  as  to  a  decrease  in  Cl  between  Fall 
1970  and  Spring  1972.    Assuming  that  without  training  El 
and  E2  would  have  experienced  similar  decreases,  we 
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concludo  that  training  had  a  positive  off.  ct  in  both 
Expcrinontal  States. 

I  torn  ^i3,  RolG  of  Planning:    how  much  nondod,  was 
not  difforont  from  cither  Experimental  ntate  in  relation 
to  the  control  cither  before  or  after  training,  and  no 
training  iffects  appeared.    The  training  program  did  not 
change  top  adninistrators'  views  of  how  much  planning  v;as 
needed  in  the  organization. 


Item  ^4,  Pole  of  Planning;    emergence,  shows  a 
dynamic  similar  to  that  observed  for  Item  #2.    Mo  state 
was  significantly  more  aware  than  others  about  the  emergence 
of  planning  at  any  point  in  time.    Neither  did  any  State 
change  over  time  (Fall  1970  to  Spring  1972) .    Ml  i'tatcs 
wore  equally  aware  of  the  variable  before  and  after  training. 

El  saw  planning  as  emerging  about  the  same  tine  as 
did  CI  before  training;  no  significant  differences  were 
measured.    However,  in  two  post-training  periods  (Pall 
1971  and  Spring  1972) ,  planning  was  considered  a  more  recent 
development  in  CI  than  in  El;  this  difference  increased  as 
time  passed.    Before  training,  it  was  felt  that  training 
was  a  more  recent  development  in  E2  than  in  CI.  After 
training  (in  Spring  1971,  Pall  1971,  and  Spring  1972)  this 
difference  ceased  to  be  significant.    El  still  put  more 
emphasis  on  this  item  during  Spring  1972  than  did  E2;  El 
viewed  planning  as  a  longstanding  practice. 

These  improvements  of  the  relative  positions  of  El 
and  E2  compared  to  CI  were  not  due  to  a  feeling  in  Experi- 
mental states  that  planning  had  been  around  longer,  but 
to  the  Control  States'  feeling  that  planning  was  a  greater 
novelty  than  they  had  reported  earlier.    If,  as  in  previous 
items,  we  presume  that  El  and  E2  would  also  have  seen  plan- 
ning as  a  newer  practice  had  they  not  taken  the  a:1A  training, 
as  CI  did  not,  then  the  fact  that  El  and  E2  stayed  about 
the  same  while  CI  decreased  is  a  positive  effect  of  training. 


Item  #5:    Fall  1970  to  Spring  1971 

Item  #5,  a  questionnaire  item,  As  I  see  it  planning 
J-s  an  integral  part  of  running  the  atate"  s  schools, 
revealed  no  change  which  can  be  attribut^id  to  training  in 
either  of  the  Experimental  organizations.    In  terms  of  the 
extent  to  which  planning  played  an  integral  role,  there 
was  a  difference  between  each  of  the  Experimental  States 
and  the  Control.    Organization  El  saw  planning  as  more 
integral  than  did  the  Control,  and  organization  E2  saw 
planning  as  less  integral  than  the  Control. 


215 


Iten  {f5:     Spring  1971  to  Fall  1971 

Analysis  of  this  iten  betwcGn  the  Spring  and  Fall 
of  1971  reveals  a  positive  training  effect  in  El  and  no 
effect  at  all  in  E2.    All  States  felt  that  planning  was 
more  integral  as  time  passed.    The  differences  over  time 
were  not  significant  in  the  case  of  E2  although  planning 
remained  less  important  there  than  in  CI.    Planning  in  El 
became  a  more  integral  part  than  in  CI  and  the  rate  of 
upward  movement  was  faster  in  the  Experimental  State  than 
in  the  Control  State^    Hence  a  positive  effect  of  training 
may  be  shown  for  El. 


Item  #5;     Fall  1971  to  Spring  1972 

Negative  effects  were  measured  in  both  Experimental 
States  as  a  result  of  AIIA  training.    El  and  E2  v/ere  less 
sure  that  planning  was  an  integral  part  of  running  the 
State's  schools  than  they  had  been  previously  or  than  was. 
either  Control  State.    El  and  E2's  confidence  in  the  impor- 
tance of  planning  decreased  more  quickly  than  CI  or  C2 
although  significant  differences  appeared  in  only  one 
comparison;    E2  with  CI. 


Item  f5;    Fall  1970  to  Spring  1972 

El  and  E2  felt  that  planning  was  not  as  important 
for  the  State's  education  system  in  Spring  1972  as  they 
had  before  training  in  Fall  1970.    CI  also  declined  over 
this  period.    A  comparison  of  the  rates  of  change  over 
this  longitudinal  period  indicates  negative  effects  in  El  and 
E2. 


Item  #6;    Fall  1971  to  Spring  1972 

Item  #6,  As  I  see  itt  persons  in  this  organization 
put  a  lot  of  effort  into  planningl     Between  Fall  1971  and 
Spring  1972,  the  amount  of  actual  work  invested  in  the 
planning  function  declined  in  the  Experimental  States  and 
the  Control  States.    In  all  cases,  this  decline  was 
significant.    Looking  at  all  four  States  in  T3,  the  greatest 
effort  was  made  by  El  follovred  by  C2,  E2,  and  CI  respectively. 
These  rankings  changed  in  T4.    While  El  still  led  the  field, 
CI  and  then  C2  were  next,  with  E2  a  distant  fourth.  We 
must  reemphasize  here  that  1)  these  inter-state  differences 
with  insignificant  and  2)  due  to  the  lack  of  pre-training 
data,  the  extent  to  which  training  effects  can  bo  assumed 
is  limited. 
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Item  f?7:    Fall  1971  to  Spring  197? 

Dositivc^rJf'       capability  to  plan  off^ctivelv  will 
m!!^u       r  7^  future  career  in  thi.g  orqanLatinn. 

sg^^  rnnn^.r-^'^  g"^?^^^  ^"         Experimental  states 

r^L^.  between  good  planning  and  a  successful 

ccareer.    This  variable  is  a  satisfactory  predictive 
mechanism  for  future  planning  success  ol  failire'since 
IffnT^T  th'^^J^^^^tional  administrators  wiJI  dcvoJe  much 
effort  to  something  they  believe  irrelevant  to  their  oraani- 

olainJna  .nd"a%''  ''^^  connections  made  iltSS^n" 

fo?  "t^p?      \  tl''''''  ""^^  '"^^^  tenuous  in  T4  than  in  T3 
r^  y  ^^'^         Control  States.    Comparing  El  with 

brtw^  i%Kr  J"""  -'^^  significant.    The  interrelationship 
botwoun  the  planning  function  and  personal  future  weakened  in 
Sr?rSas'th^°.'^  States  although  only  in  the  casf  " 

IL^lnJl  ^  Significant.    E2  also  saw  the  slimmest 

connection  in  both  periods. 

Item  #8;    Fall  1971  to  Sprinc^  1972 

"^'"^f^,  The  activities  relatincy  to  planning  are 
having  an  effect^h  the  policy  oi  thic,^nrn^!ni^:>^irl  ir. 

Tilth.lV'T  °^  ty.^^^^ib^iity  oi  the  glannin^  TOs 
whxch  tests  how  much  influence  planning  exerts  upon 
c^!«"i''?^^°"^^'^''°^^^°"^^^^"9.    If  little  connection  is 
fhS  ™  '""^  ^°t"^^  policy,  persons  within 

^^''^^y  t°  ^^^^  planning  as  busyrvork 
-f?       essential  management  tool.    We  were  con- 

on^?n^  TJi?  •^''^^f!       ^^^^  variable  between  Fall  1971  and 

opring  1972  in  both  Experimental  States.    The  connoction 

Of  planning  to  policy  making  declined  significantly  in  E2 

but  was  not  sufficient  to  create  differences  between  it 

o?  n??n^^«2"^''°^  the  greatest  influence 

of  planning  on  policy  in  T3  and  in  T4.    The  significant 
difference  between  this  Experimental  state  and  the  Control 
states  was  more  a  function  of  stability  than  of  change 
Z'il    i"?*^'    5i  started  and  finished  by  makina  the  closest 
connection;  E2,  on  the  other  hand,  immediately  followed  El 
in  T3  but  fell  to  last  in  T4.    It  seems  particularly 
significant  that  a  State  with  substantial  training  rnd 
involvement  in  planning  could  believe  that  planning  and 

S?iins  whfSh^K''?  distantly  connected  than  did  two  Control 
States  which  had  far  less  planning  experience. 
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DATA  SUrH-IARY 

The  Credibility  of 
the  Planning  Process 


Fall.  1970  -  spring,  1972  TRAINING 


Item 

Type  of  Data 
CONTENT 

Positive 
Effect 

No 
Effect 

Negative 
Effect 

1 

Establish  credibility  of 
planning 

El 

E2 

2 

Koxc  or  irXannin^*  now 
integral 

E1,E2 

3 

Rolo  of  Planning:    how  much 
is  nocdod 

E1,E2 

4 

Role  of  Planning:  cmGrgcnco 
QUESTION^L•^IRE 

E1,E2 

5 

As  I  SCO  it^  olc^nning  is  (in 
integral  part  of  running  the 
state's  schools. 

F1/F2 

6 

As  I  soo  it,  persons  in  this 
orgrinization  put  a  lot:  of 
effort  into  planning. 

F1,E2 

7 

capability  to  plan  effec- 
tively v;ill  positively 
affect  my  future  career  in 
this  organization. 

E1,E2 

8 

The  activities  rel?.ting  to 
planning  are  having  an  effect 
on  the  policy  of  this  organi- 
zation. 

El 

E2 

Credibility  of  organizational  plann:.ng  among  top 
administrators  was  significantly  affected  by  the  ATUl  trail 
ing  program.    Out  of  the  eight  variables  (4  content  cate- 
gories and  4  questionnaire  items)  which  constitute  our 
analysis  of  the  credibility  of  the  planning  process,  only 
one  (Item  #3)  exhibited  no  training  effects  at  all. 
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^t.    ,  .       evaluation  of  tha  content  data  indicated  that 
the  ATIA  program  furthered  the  credibility  Sf  planning  in 
^^emf  ?hn,nrJ^'  The  me^aning  of  the  fi?s?  Sf  Sese 

Items  should  be  regarded  cautiously  due  to  the  lack  of  a 
control  group  and  the  consequent  problem  of  validity  of 

Nevertheless,  AMA  training  established 
the  credibility  of  planning  about  as  solidly  as  El  had 
?^^?pSl?!'''f  to  training,  but  not  so  firmly  as  E2  wanted. 
In/io  l    ^-^^  attitudes  about  the  role  of  planning,  El 
•  experienced  positive  effects  in  making  planning  a 
more  integral  part  of  running  the  State's  schools  (Itim  #2) 
as  well  as  a  practice  of  longer  standing  (Item  #4).  Both 
Experimental  States  attached  similar  levels  of  importance 
to  planning  before  and  after  training    -.o  no  effects  were 
attributed  to  the  AMA  on  Item«3. 

On  the  other  hand,  more  negative  than  positive 

?hn^S^%S?P?^''S^  reviewed  the  questionnaire  items, 

though  the  lack  of  pre- training  information  prompted  us 
to  be  careful  in  assigning  these  effects.    The  Experimental 
States  viewed  planning  as  less  integral  (Item  #5)  and  as 
an  activity  to  which  organization  members  devoted  less 
effort  (Item  #6)  as  tiinta  passed.    Furthermore,  the  influence 
of  planning  on  policy  (Item  #0)  within  E2  seemed  to  diminish 
as  a  consequence  of  planning;  El's  sense  of  its  influence 
was  unchanged.    The  single  positive  training  effect  was 
the  firm  connection  made  between  planning  skills  and  career 
development;  both  States  thought  the  connection  had  been 
strengthened  by  training. 

The  emergent  patterns  of  questionnaire  items  in 
earlier  sections  appear  again  here.    On  the  basis  of  mean 
scores,  E2  generally  thought  planning  was  least  credible, 
while  FL  believed  th?'^  planning  had  greatest  credibility 
in  their  organization. 

NO  overall  conclusion  is  made  as  to  the  total  inoact 
of  training  on  the  credibility  of  the  olanning  process/ 
The  evidence  is  mixed  and  somewhat  contradictory  (e.g 
"^f^r^?^?^         integral  a  rol^  planning  plays).    Our  assess- 
ment of  training  effects  on  planning  credibility  must  rest 
on  an  individual  analysis  of  the  various  items  which  com- 
prised the  section.    On  this  evidence  the  reader  may  or 
may  not  wissh  to  drav;  overall  conclusions. 
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AREA  Hit     LOCAL  EDUCATIONAL  AGENCY  DATA 

The  purpose  of  this  section  is  to  study  the  impact 
of  ATIA  training  on  the  Experimental  Local  Education 
Agencies  (LEA's)  in  terms  of  the  causal  variables  discussed 
earlier. 3o    We  again  look  at  the  thirteen  criteria  listed 
by  the  AMA  as  the  basis  of  evaluating  the  program  (Expected 
Results). 37    The  extent  to  which  the  AIlA  was  able  to  work 
through  the  LEA's  to  achieve  each  goal  is  then  considered 
(Actual  Results).    VJhat  participants  think  about  what  AflA 
did  is  summarized  emd  displayed  in  tabular  form  (Attitudinal 
Data) .    Finally,  we  offer  conclusions  which  rest  on  a 
combination  of  results  and  attitudes. 

Besides  using  the  scaled  eind  open  response  question- 
naires as  information  resources  for  this  section,  we 
examined  the  original  planning  documents  produced  at 
Hamilton. 38    From  these  sources  wo  sought  a  rounded  appraisal 
of  the  status  of  the  thirteen  criteria  applicable  to  the 
lea's  which  underwent  AIlA  training. 


The  symbols  usad  here  to  denote  Experimental  and 
Control  local  education  agencies  arc  the  same  as  diagrammed 
in  an  earlier  section: 


S 

y 

M 
B 
0 
L 
S 

U 

s 

E 
D 


ELEA  #1  =  experimental  local  education  agency  In 
same  State  as  ESED  #1 

ELEA  #2  =  experimental  local  education  agency  in 
same  state  as  ESED  #1 

CLEA  #1  =  control  local  education  agency  in  same 
state  as  ESED  #1 

ELEA  #3  =  experimental  local  education  agency  in 
same  state  as  ESED  #2 

ELEA  #4  =  experimental  local  education  agency  in 
same  state  as  ESED  #2 

CLEA  #2  =  control  local  education  agency  in  same 
state  as  ESED  #2 


37 


38 


A?1A,  0£.  cit.  /  pp.  4-5. 

Klawuhn  and  Basso,  op. 

ELEA  #1  *  Vol.  IV 

ELEA  #2  =  Vol.  V 

ELEA  #3  =  Vol.  II 

ELEA  #4  =  Vol.  Ill 


cit. /  Appendices. 
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Section  1:  Action 

A.    Expected  Results 

To  facilitate  our  review,  we  repeat  the  thirteen 
criteria  previously  described.    Refer  to  the  SED  Expected 
Results  section  for  more  detailed  definitions  of  terms. 

1.  agreed  upon  a  definition  of  the  institution's 
mission; 

2.  established  continuing  ob')octives  and  planning 
procedures  for  long  range  achievement  of  the 
institution's  mission; 

3.  identified  resources  cind  constraints; 

4.  differentiated  between  where  the  institution 
is  and  where  it  wcints  to  go; 

5.  modified  previously  established  objective.-:^; 

6.  identified  and  analyzed  alternative  courses 
of  action; 

7.  determined  priorities; 

8.  made  strategic  action  assignments; 

9.  defined  standards  of  performance  for  key 
administrators; 

10.  specified  task  completion  dates ; 

11.  designed  supplementary  planning  efforts; 

12.  assigned  r espons ib i 1 it ies  to  subordinate  units; 

13.  designed  a  methodology  by  which  future  performance 
may  be  eveluated  in  relation  to  the  performances 
specified  in  the  plan. 


B.    Actual  Results 

We  will  address  the  question  of  what  has  been  done 
in  each  LEA  to  satisfy  each  criterion.    Since  we  did  not 
interview  participants  nor  collect  as  much  written  informa- 
tion as  from  the  SED's,  our  analysis  will  be  shorter  and 
more  tentative  than  the  one  presented  for  the  SED's. 
Enough  data  is  available  to  draw  an  outline  of  LEA  progress 
and  to  give  some  indication  of  future  directions. 
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It  is  well  to  rG-emphasize  here  that  it  is  not 
within  the  scope  c/  this  evaluation  to  examine  the  feasibil- 
ity or  relevance  of  what  the  agencies  have  written.  Assump- 
tions, beliefs,  missions,  etc.  are  largely  personal 
statements  beyond  empirical  verification.    Other  elements 
are  based  on  in-house  data  beyond  reasonable  external 
analysis.    Our  purpose,  therefore,  is  simply  to  determine 
to  what  extent  participants  did  what  AMA  said  they  would 
do.    To  put  it  more  bluntly,  we  wish  to  see  if  paper  was 
produced  in  correspondence  to  the  thirteen  criteria. 

Consequently,  we  have  devised  the  following  "Summary 
of  Action  Findings:    LEA  Level"  chart  to  assess  progress 
toward  each  criterion.    To  keep  within  the  scope  of  this 
evaluation,  consideration  of  progress  is  limited  to  quanti- 
tative rather  than  qualitative  concerns.    When  we  indicate 
"minimum,  "moderate,"  or  "maximum"  progress  this  is  not 
to  be  interpreted  as  "poor,"  "average,"  or  "excellent. 
It  should  be  interpreted  as  how  much  work  was  done  on  a 
criterion,  not  how  good  that  work  may  have  been. 
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SUMMARY  OP  ACTION  FINDINGS  39 
LEA  LEVEL 
AHA  CRITERIA  1-13 
FALL,  1970  to  SPRING,  1972 

Amount  o£  Progress 


H2jl  Criteria  Minimum     Moderate  Maximum 

1.  Agreed  upon  a  definition  of  ELlrEL2 
the  institution's  mission  EL3rEL4' 

2.  Established  continuing 
objectives  and  planning 
procedures  for  long-range 

achievement  of  the  insti-  ELI  FL2 

tution's  mission  EL3^FL4' 

3.  Idontifie4  resources  and  ELlrEL2y 
constraints  £^3  ^  ' 

4.  Differentiated  between 
whore  the  institution  is 

going  and  where  it  wants       EL3  ELlrSL2r 
to  go  EL4 

5.  Modified  previously  estab-  ELlrEL2r 
lished  objectives  EL3^EL4 

6.  Identified  and  analyzed 
alternative  courses  of 

action  EL3  EL1,EL4  EL2 

7.  Determined  priorities  ELlrEL2r 

EL3,EL4 

8.  Made  strategic  action 

assignments  EL3  EL2rEL/J  ELI 

9.  Defined  standards  of  per-  ELl^ 
formance  for  key  adminis-  EL2r 
trators.  EL3yEL4 

10.  Specified  task  completion 

dates  ELI ,EL3      EL2 , EL4 

11.  Designed  supplementary  ELl^ 
planning  efforts  EL3  EL2rEL4 

12.  Assigned  responsibilities  ELlrEL2^ 
to  subordinate  units  EL3rEL4 

13.  Designed  a  methodology  by 

which  future  performance  ELlrEL2 
may  be  evaluated  in  re la-  EL3^EL4 
tion  to  the  performances 
specified  in  the  plan 

41  39 
gj^^  On  all  charts  used  in  the  action  and  attitude 
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In  terms  of  activity  directed  toward  the  accomplish- 
ment of  the  AflA  training  goals,  the  Expeririental  LEA's 
took  some  steps  toward  nearly  every  objective.    Led  through 
the  step-by-step  structured  planning  exorcise,  they  ad- 
dressed all  the  criteria  enumerated  in  the  original  AMA 
proposal. 

The  greatest/  most  uniform  progress  v;as  made  on  the 
first  and  second  criteria.    The  institutions'  missions  were 
defined,  and  a  set  of  continuing  objectives  were  written. 
Each  mission  and  continuing  objectives  series  was  somewhat 
different;  some  documents  were  more  detailed  than  others. 
The  essential  point  is  that  the  mission  and  continuing 
objectives  which  AMA  promised  to  produce  were,  in  fact, 
written. 


Comparable  degrees  of  progress  toward  criteria  3  and 
4  were  made  in  the  Experimental  LEA's,  which  went  to  con- 
siderable lengths  to  analyze  their  environment;  part  of 
this  self-examination  was  some  identification  of  resources 
and  constraints .    Implied  in  their  examinations  was  an 
analysis  of  where  the  Institution  is  ^oing  and  where  it 
wants  to  go. 

As  with  most  of  the  elements  of  the  planning  process 
experienced  by  che  teams,  the  degree  to  which  particular 
criteria  or  topics  are  developed  depends  heavily  upon  the 
interests,  abilities,  and  data  available  to  the  teams. 
CLEA  #2  generated  all  kinds  of  data  on  pupil  achievement, 
budgetary  trends,  etc.    ELEA  #4  seemed  to  develop  a 
thorough  list  of  cost  reduction  strategies  based  on  avail- 
able resources*    Conversely,  ELEA  #3's  minimtun  rating  was 
dictated  by  the  relatively  slight  documentation  they  had 
on  the  current  status  of  rheir  programs  and  institutional 
environment. 

The  modification  of  j^reviously  established  objectives 
was  accomplished  by  the  development  of  specific  objectives 
(criterion  5).    These  specific  objectives  were  written  to 
correspond  to~and  be  grouped  under— the  continuing  objec- 
tives.   In  many  instances,  a  set  of  specific  objectives 
did  not  seem  to  have  been  completed  for  the  continuing 
objective (s) .    But  the  expected  results,  client  groups, 
and  deadlines  were  clearly  evident  in  those  which  wore 
completed. 

Strategies  contribute  to  the  achievement  of  specific 
objectives  ana  are  covered  by  the  sixth  criterion  (identified 


sections  to  refer  to  LEA's,  these  short-form  symbols  will 
be  used: 

ELEA  #1  ELI  ELEA  #3  =  EL3 

ELEA  #2  =  EL2  ELEA  #4  «  EL4 

CLEA  #1  =  CLl  CLEA  #2  -  CL2 


224 


and  analyzed  alternative  courses  of  action) •    They  often 
appeared  in  a  standardized  format: 

Area  Strategized  (Objectives,  Strengths,  Weaknesses, 

Planring  Gap,  etc.) 

Strategy:  Costs: 

Program:  Manpower: 

Task^-Action  Assignments     Responsibility       Due  Date 


Just  as  every  continuing  objective  did  not  list  all  of  the 
specific  objectives  necessary  for  its  attainment,  so  every 
specific  objective  was  not  fully  strategized.  Strategies 
demand  more  specificity  than  the  specific  objectives  and 
involve  numerous  hard  decisions  which  may  have  been  post- 
poned until  managers  returned  to  their  organizations  after 
training.    Strategies  cannot  be  systematically  approached 
until  comparatively  late  in  the  sessions;  time  may  have 
run  out.    This  dual  factor  may  account  for  the  underdevelop- 
ment of  strategies. 

ELBA  #2's  planning  document  offered  the  most  ccwnplete 
connections  between  specific  objectives  and  strategies. 
More  of  their  objectives  were  fully  strategized  than  those 
in  other  plans  we  reviewed.    Moderate  but  highly  acceptable 
progress  was  made  in  ELBA  #1  and  #4;  ELBA  #3  was  not  so 
attentive  to  strategies  as  the  others. 

None  of  the  LEA's  clearly  delineated  their  priorities 
(criterion  7)  in  the  documents  made  available  to  the  research 
teeun.    In  ELBA  #1,  we  could  only  presume  priorities,  since 
the  manner  of  presentation  differed  for  each  suspected 
priority  and  was  confusing. el  #3  and  #4  weighted  their 


ERIC 


40 

In  the  AT^A  planning  document  for  ELBA  #1,  priorities 
are  typically  presented: 


PRIOKITISS 

Elementary 

Priority  Reading  Strategies 

1.    Establish  data  base  of 
student  performance  and 
eJDility 


PRIORITIES--SECONDARY 

Specific  Ob jectivejs— Prepare 
for  continued  educiation 

Priority  Strategies 

1. 


2.    Lead  teachers  in  elementa- 
ry schools  to  improve 
reading   2. 


Personnel  needs  (Subjective 
eval . ) 

a)  administrative 

b)  teacher 


Preparation  for  entry  jobs.... 
Reading  this  over,  one  could  reasonably  assume  that  ELBA  #1  has 
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continuing  objectives  by  the  assignment  of  points  but  did 
not  specify  the  cutoff  level  for  priority  status.  By 
scanning  a  list  of  continuing  objectives  with  descending 
weighted  averages  the  reader  cannot  learn  whether  three 
continuing  objectiveS/fouRror  some  other  number  have  been 
termed  priorities.    In  all  cases  a  simple  listing  of  issues 
identified  as  district  priorities  would  have  been  clearest. 

Action  assignments  (criterion  8)  cover  the  creation 
of  job  descriptions  for  key  personnel  in  the  organization* 
Job  descriptions  sometimes  include  performance  standards 
(criterion  9)  and  task  completion  dates  (criterion  10)/ 
to  be  discussed  later.    Here  we  evaluate  the  definition  of 
the  position  and  a  listing  of  responsibilities  of  that 
position.    ELBA  #3  was  unable  to  comply  with  this  training 
goal;  ELBA  #2  and  ELBA  #4  did  some  work  on  it;  job  descrip- 
tions were  a  particular  strength  of  the  ELBA  #1  planning 
document. 

Another  area  of  minimal  progress  was  criterion  9, 
standards  of  performance*    While  responsibilities  (action 
assignments)  were  plainly  listed  throughout  most  planning 
documents^  expected  results  (performance  stemdards)  were 
not.    No  direct  connection  linked  responsibilities  to 
results.    Since  only  one  Experimental  SED  has  made  sub- 
stantial progress  in  this  regeird,  the 

unimpressive  achievement  of  the  local  level  is  unsurprising 

Efforts  to  specify  task  completion  dates  (criterion 
10)  for  action  assignments  met  similar  success  in  the  ELBA* 
Mciny  of  the  decisions  made  by  trainees  had  firm  deadlines 
attached  to  them.    ELBA  #2  and  ELBA  #4  apparently  placed 
great  in^ortance  on  this  criterion.    ELBA  #1  cind  ELBA  #3, 
for  whatever  reasons,  did  not  appear  so  concerned  to  estab- 
lish completion  dates  for  tasks  assigned  at  Hamilton. 

Criteria  11  and  12  were  interpreted  to  suggest 
activities  such  as  in-service  training,  planning  guidelines 
etc.    We  have  already  mentioned  ELBA  #2's  presentation  of 
current  achievement  information  in  their  planning  document* 
Both  of  these  criteria  are  so  broadly  stated  as  to  defy 
precise  definition.    For  this  reason,  the  research  team 
has  taken  refuge  in  the  "moderate**  column  for  nearly  all 
BLEA'fi.    All  produced  reports  and  paper  decisions  which 
could  be  taken  as  supplementary  planning  efforts  or  the 


set  reading  as  its  elementary  schools  priority  and  career/ 
higher  education  as  its  secondary  schools  priority.  But 
the  disparity  of  form  and  content  for  the  two  levels  as 
well  as  lack  of  clarity  make  our  priority  designation 
difficult. 
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assignment  of  responsibility  to  subordinate  units.  Using 
this  s.^e  decision  rule,  since  ELBA  #3  did  not  insert  .as 
much  information  about  these  criteria  in  their  planning 
document,  a  minimum  rating  was  assigned  to  their  supple- 
mentary planning  efforts. 

Evaluation  (criterion  13)  was  another  weak  point  in 
the  AHA  program  in  the  LEA's.    Like  their  ESED  counterparts # 
the  lea's  have  not  developed  an  adequate  evaluative  system 
based  on  the  plan  produced  through  AMA  training.    Some  of 
the  objectiyes  suffer  from  overgeneralized  evaluation 
strategies. Others  fail  to  list  any  means  of  evaluation 
at  all. ^2    Where  specific  indices  do  appear,  a  suitable 
testing  program  is  strongly  recommended.    This  is  under- 
standable given  the  traditional  reliance  of  American 
education  on  this  device;  especially  in  assessing  reading 
skills,  testing  had  obvious  utility. 

The  SED's  made  some  progress,  albeit  minimal,  in 
broadening  their  evaluation  base  beyond  simple  paper-and- 
pencil  tests;  the  LEA's  seem  to  have  every  written  intention 
of  following  suit.    Some  acceptable  examples  of  evaluation 
strategies  were  found  among  the  ELBA  #3  materials.  Evalua- 
tion certainly  concerns  the  ELEA's,  for  many  objectives 
and  strategies  of  evaluation  appeared  throughout  the 
planning  documents.    In  the  Spring  of  1972  the  research 
team  requested  that  the  Experimental  ELEA's  forward  to  us 
their  current  plans  and  an  up-to-date  status  report  regard- 
ing progress  on  the  plans  produced  during  AMA  training. 
Responses  to  that  request  are  discussed  in  Chapter  Six. 
As  v/e  have  indicated,  any  meaningful  evaluation  of  the 
effectiveness  of  AMA  training  must  be  based  on  an  evalua- 
tion which  looks  at  attitudes  and  actions  over  time  and  their 
effect  on  organizational  output. 


Section  2:  Attitudes 


After  considering  whether  LEA's  accomplished  the 
thirteen  training  goals  of  the  AI-IA,  we  must  look  at  how  they 
view  their  accomplishments.    Attitudes  expressed  about  various 
planning  activities  at  the  Hamilton,  New  York  training  site 


An  example  can  be  drawn  from  ELEA  #2:  "Identify 
seventh  grade  students  reading  belou'  sixth  grade  level  on 
standard  achievement  test." 

^^An  example  can  be  drawn  from  ELEA  #4:    "By  June 
1972,  7th  and  10th  grade  students  will  demonstrate  the 
ability  to  accept  a  new  school  environment  as  measured  by 
Q  ^appropriate  means." 
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foreshadow  the  future  development  of  the  team  planning 
process  after  participants  returned  to  their  organization. 

Attitudinal  data  for  this  section  is  divided  into 
four  basic  areas: 

A.    Development  of  Organizational  Mission  and 
Objectives 

Mobilization  of  Organizational  Planning 

C.  Top  Management  Support  for  Planning 

D.  Credibility  of  the  Planning  Process 

Data  was  gathered  from  participating  LEA's  through 
questionnaires  alone.    In  displaying  questionnaire  data 
within  each  area^  two  comments  will  be  provided: 

1.  Questionnaire  Item:    The  statement  in  the 
questionnaire  to  which  a  response  was  given 

on  a  scale  of  (1)  not  at  all  to  (7)  very  often. 

2.  Points  of  Time:    For  each  item  the  time  in 
which  it  was  administered  will  be  indicated 
(Tl  =  Fall  1970?  T2  =  Spring  1971;  T3  =  Fall 
1971;  and  T4  «  Spring  1972) . 

In  most  cases r  training  effects  were  deteznnined  on 
the  basis  of  a  T1-T4  comparison.    The  research  ti^am  is  most 
confident  of  these  conclusions.    However ,  due  to  recent 
additions  of  questionnaire  items,  it  was  sometimes  necessary 
to  assign  effects  after  a  T3-T4  comparison  alone.  I'Jhile 
the  absence  of  T1-T2  data  counseled  cautious  interpretations , 
v/e  are  certain  our  decisions  were  valid.    This  is  especially 
true  where  we  obtained  organizational  documents  to  support 
other  analysis.    Furthermore^  the  T3-T4  ccxnparisons  also 
include  Control  LEA*s  vrhich  were  not  included  in  T1-T2. 
The  presence  of  such  multiple  m(oasures  provides  double 
coverage  on  many  items.    For  these  reasons f  we  felt  justified 
in  attributing  positive  effects/  no  effects f  or  negative 
effects  on  T2-T4  as  well  as  T1-T4  comparisons. 

Only  the  Experimental  LEA's  responded  in  Tl  and  T2. 
In  T3  and  T4,  tv;o  Control  LEA's  were  added  to  the  analysis. 
Therefore f  most  T3-T4  cosnparisons  include  Experimental  and 
Control  LEA*s.    The  single  exception  to  this  occurs  on  items 
which  v;ere  designed  to  measure  attitudes  toward  AMA*s  role 
in  the  training.    Since  questions  about  the  AMA  could  not 
be  asked  of  organizations  which  did  not  participate  in  the 
program^  the  Control  LEA's  were  not  asked  to  respond  to  them. 
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A«    Development  of  Organizational  Mission  and  Objectives 

Seven  perspectives  on  the  question  o^  defining 
organizational  missions  and  objectives  are  provided  by 
seven  questionnaire  items: 

1.  The  goals  of  this  organization  are  articulated 
Points  of  Time:    Tl,  T2,  T3,  T4 

2.  Our  goals  are  realistic  and  attainable  with 
our  best  efforts^ 

Points  of  Time:    Tl,  T2,  T3,  T4 

3.  The  top  priority  objectives  for  state  educa- 
tion are  clear  to  me> 

Points  of  Time:    T3,  T4 

4.  I  feel  that  the  objectives  developed  during 
aFia  training  reflect  the  most  serious  and 
pressing  needs  of  state  education^ 

Points  of  Time:    T3,  T4 

lea's:     ELI,  EL2,  EL3,  EL4 

As  this  pertains  specifically  to  MiA  train- 
ing it  was  not  asked  in  the  Control  LEA's 
(CLl  and  CL2) 

5.  As  I  see  it/  the  operational  priorities  of 
the  objectives  developed  during  AMA  training 
are  clear > 

Points  of  Time:    T3,  T4 

lea's:     ELI,  EL2,  EL3,  EL4 

As  this  pertains  specifically  to  AMA  training 
it  was  not  asked  in  the  Control  LEA's  (CLl 
and  CL2) 

6.  The  kinds  of  things  I  am  doing  will  make  a 
long  term  contribution  to  education > 

Points  of  Time:    T.l,  T2,  T3,  T4 

?•    As  I  gee  it,  my  organization  is  moving  in  the 
right  direction  >  " 

Points  of  Time:    T3,  T4 
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Itm     3       The  top  priority  objectives  for 
state  education  are  clear  to  me. 


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

X 

SD 

Experimental  LEA  #1 

1.218 

^1 

5.161 

1.128 

Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 

a.R7n 

1  . 

3? 

5.093 

1.422 

?4 

4.416 

1  . 

23 

4.000 

1.167 

29 

4.S17 

27 

4.444 

1.908 

30 

?,0 

5.:^oo 

Control  LEA  #2 
Total 

d.  m 

*T  «  «J  V 

X  «  O  X  o 

182 

166 

Two  Way  Analysis  of 
Variance 

F 

Sisnif . 

{Experimental  LEA#1W/ 

0.084 

NS 

lExperimental  LEA#2 

Row 

0.937 

NS 

Experimental  LEA#3W/ 

Q9l, 

0.647 

NS 

Experimental  LEA#4 

Row 

0.803 

NS 

Experimental  LEA#1W/ 

Col. 

0.000 

NS 

pontrol  LEA  #1 

.Row 

0.040 

MS 

Experimental  LEA#2W/ 

Col. 

0.431 

NS 

Pontrol  LEA  #1 

Row 

1.186 

nS 

Experimental  LEA#3W/ 

Col. 

0.115 

MS 

Control  LEA  #2 

Row 

0.014 

MS 

Experimental  LEA#4W/ 

Col. 

0.101 

■  ■    MS  ■■ 

Control  LEA  #2 

Row 

0.745 

MS  ■■ 
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4   I  feel  that  the  objectives  developed  during  AMA 

training  reflect  the  most  serious  and  pressing 
needs  of  state  education. 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4  . 

Total 

N 
32 

X 

5.843 

SD 

0.91C 

N 

31 

X 

5.806 

SD 

1.077 

31 

5.612 

1.406 

32 

5.593 

1.240 

2A. 

4.458 

1.74^ 

21 

5.000 

1.483 

22. 

.1.575 

26 

5.076 

1.622 

in 

111 

Two-Way  Analysis  of 
Variance 

T3  §  T4 

Experimental  LEA#1  W/ 
fixnerimental  LEA#2 

F 

Signif . 

0.018 

NS 

Row 

1. 123 

NS 

Experimental  LEA#3  W/ 
Experimental  LEA#4 

1.120 

NS 

Row 

0.715 

NS 
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5  As  I  see  it^  the  operational  priorities  of  the 

"objectives  developed  during  AMA  training  are  clear > 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 

N 

32 

X 

6.000 

SD 
0.879 

N 

31  ; 

X 

5.838 

SD 
l.Ofi? 

31 

5.483 

1.338 

32  ! 

5.531 

1.319 

.24, 

1.793 

21  .i 

1.952 

1-28.^ 

4.827 

1.554 

i.692 

l.()91 

Total 

IK 

111 

Two-Way  Analysis  of 
Variance 

T3   5  T4 

Ex?3erimental  LEA#1  W/ 
Experimental  LEA#2 

F 

Signif . 

Coll 

0.075 

NS 

Row 

3.927 

.05 

Experimental  LEA#3  W/ 
Experimental  LEA#4 

0.241 

NS 

Row 

0.010 

NS 
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Item  #1;     Fall  1970  to  Spring  1971 

Item  #1/  The  goals  of  this  organization  are  articu- 
lated/  is  a  comparison  of  Experimental  LEA  #1  (ELI)  with 
Experimental  LEA  #2   (EL2)   and  Experimental  LEA  #3  (EL3) 
with  Experimental  LEA  #4  (EL4) ♦     Absence  of  any  Control 
LEA  prompts  caution  about  ascribing  training  effects  to 
the  AMA  program.     But  an  analysis  of  the  data  reveals  that 
the  goals  of  all  four  Experimental  LEA's  were  less  articu- 
lated in  Spring  1971/  after  training  had  ended/   than  they 
had  been  in  Fall  1970/  before  training  had  begun.  In 
neither  comparison  was  there  any  difference  between  the 
lea's  but  EL2  did  decline  much  more  sharply  than  the  rest. 
And  since  this  decline  was  significant/  negative  training 
effects  can  be  attributed  to  EL2,     The  losses  in  goal 
articulation  in  the  other  LEA's  were  insufficient  to 
warrant  similar  action. 

Item  #1;     Spring  1971  to  Fall  1971 

Except  for  EL2/  the  AMA  program  had  no  effect  on 
the  lea's  between  Spring  1971  and  Fall  1971.     In  contrast 
to  the  preceding  period/  all  four  groups  increased  the 
degree  to  which  they  felt  their  goals  were  articulated. 
These  increases  were  similar  enough  not  to  produce  any 
differences;  no  LEA  saw  the  goals  as  significantly  more 
articulated  than  other  LEAs.     However/  the  rise  in  per- 
ceived goal  clarity  was  a  major  change  for  EL2.  So 
instead  of  being  adversely  affected  by  training  as  was 
true  in  T1-T2/  EL2 ' s  level  of  goal  articulation  was  raised 
by  the  AMA  program. 

Item  #1;     Fall  1971  to  Spring  1972 

The  addition  of  Control  LEA  #1   (CLl)   and  Concrol 
LEA  #2   (CL2)  to  each  set  of  comparisons  will  strengthen  our 
appraisal  of  results  of  the  AMA  change  strategy.  Review- 
ing events  in  the  LEA's  between  Fall  1971  and  Spring  1972/ 
we  cannot  attribute  any  effects  of  training.     ELI  held 
their  advantage  over  EL2  and  also  sensed  that  their  goals 
were  more  articulated  than  their  corresponding  Control  LEA 
(CLl) •     Other  than  that/  no  difference  between/  or  changes 
among,   the  LEA's  can  be  ascribed  to  their  Hamilton 
experiences* 

Item  #1:     Fall  1970  to  Spring  1972 

Our  overall  assessment  considers  change  on  a  Fall 
1970  to  Spring  1972  basis.     Studying  these  time  frames,  it 
is  evident  that  EL3  perceives  a  higher  level  of  articulation 
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for  their  goals  than  FL4  perceives  for  theirs;  although 
EL3^s  goals  are  Igss  articulated  after  training  than  before, 
while  EL4's  are  more  articulated.    Tho  absence  of  any 
definite  shifts  over  time  though  preclude  any  assignment 
of  effects  to  either  group.    Both  ELI  and  EL2  enhanced 
goal  articulation  since  training  ended;  yet  EL2's  advance 
was  considerably  less  than  ELl's.    For  that  r(^ason,  we 
conclude  that  AIIA  was  responsible  for  positive  training 
effects  in  ELI. 


Item  #2;    Fall  1970  to  Spring  1971 

The  degree  to  v/hich  The  goals  are  realistic  and 
attainable  with  our  best  efforts  is  Item  #2  in  this  sect ion . 
Here  we  contrast  tne  two  Experimental  LEA* s  in  each  state 
directly /  without  a  control  group.    On  account  of  this, 
our  amalysis  vrill  be  less  straightforward  during  this  period 
than  it  might  have  been  with  a  control  group.    One  clear 
development  is  that  all  LEA's  perceived  their  goals  to  be 
less  realistic  and  attaineible  after  training  than  before. 
The  extent  to  which  AfIA  can  bo  held  accountable  for  this 
decline,  however,  is  limited  to  ELI  and  EL2  which  both 
experienced  similar  significant  reductions.    Neither  EL3 
nor  EL4  diminished  their  evaluations  enough  to  make  any 
decisions  on  training  effects  on  those  organizations. 


Item  ft2:    Spring  1971  to  Fall  1971 

The  situation  in  T2-T3  was  altogether  different. 
AI4A  could  be  credited  with  positive  effects  in  the  four 
Experimental  LEA's.    Not  only  did  each  consider  its  goals 
significantly  more  realistic  in  the  Fall  than  in  the 
previous  Spting  but  also  more  realistic  than  in  the  Fall 
of  1970,  before  training  began.    These  T2-T3  enlargements 
in  felt  goal  feasibility  more  than  made  up  for  the  decreases 
reported  in  the  immediate    post-training  period.    Of  the 
four  groups,  ELI  registered  the  best  feeling,  although  FL2 
experienced  most  growth  over  time. 


Item  #2:    Fall  1971  to  Spring  1972 

The  T1-T2  pattern  reasserted  itself  during  this 
time  as  the  LEA's  again  dropped  in  their  estimation  of 
their  goals.    However,  these  losses  were  not  significant 
and  cannot  be  attributed  to  any  effects  of  AMA  training. 
This  no-effects  conclusion  is  reinforced  by  additional 
comparisons  with  the  Control  LEA's.    ELI  and  EL2  were  not 
different  from    nor  did  they  change  relative  to,  CLl;  the 
same  was  true  for  EL3  and  EL4  as  against  CL2.    There  were 
no  effects  due  to  implementation  of  the  AMA  program  in  any 
lea's. 


238 


Item  #2;     Fall  1970  to  Spring  1972 

Only  in  ELI  and  EL2  could  any  effects  be  linked  to 
the  training  experience*    Based  on  a  Fall  1970  to  Spring 
1972  evaluation,  we  held  that  training  helped  shape  ELI 
and  EL2's  goals,  making  them  more  realistic  and  attainable. 
While  EL3  and  EL4  also  increased  over  this  period, 
modifications  in  those  two  organizations  were  insignificant; 
nothing  in  their  cases  can  be  credited  to,  or  blamed  upon, 
MIA. 


Item  #3;    Fall  1971  to  Spring  1972 

Item  #3,  The  top  priority  objectives  for  state 
education  are  clear  to  me,  is  another  aspect  of  the  deve lop- 
ment  of  organizational  mission  and  objectives*  Reactions 
to  this  statement  were  invited  from  the  Experimental  LEA*s 
and  the  Control  LEA's  covering  the  period  Fall  1971  to 
Spring  1972.    ELI,  EL2,  EL3,  and  EL4  showed  diminished 
awareness  of  state  educational  priorities  while  CLl  and 
CL2  intensified  their  awareness •    Nevertheless  all  these 
changes  were  not  enough  to  register  significant  difference 
among  the  LEA"s.    No  training  effects  can  be  attributed  to 
AMA  on  the  clarity  of  top  state  educational  objectives  to 
local  school  officials  in  either  State. 


Item  »4;    Fall  1971  to  Spring  1972 

I  feel  that  the  objectives  developed  during  AMA 
training  reflect  the  n\08t  serious  and  pressing  needs  of 
state  education^  Item  #4,  could  only  address  the  LEA' s  wh i ch 
had  undergone  AMA  training;  no  control  group  was  possible. 
It  was  also  restricted  to  a  T3-'T4  comparison.    Increases  and 
decreases  in  felt  applicability  of  objectives  to  needs  were 
uniform  within  each  State.    There  was  a  slight  extension 
of  relevance  in  EL3  and  EL4  and  a  small  reduction  in  ELI  and 
EL2.    Neither  was  substantial;  each  was  about  the  seune  in 
the  Spring  as  it  had  been  the  previous  Fall.    Hence  no 
effects  were  present  in  the  LEA's  as  a  result  of  the  program. 


Item  <?5:    Fall  1971  to  Spring  1972 

Item  #5  asks  respondents  to  indicate  their  under- 
standing of  priorities  among  their  own  organizational 
objectives:    As  I  see  it,  the  operational  priorities  of 
the  objectivei"developed  during  TtflA  training  are  clear. 
ELI'S  reaction  was  then  matchca  with  EL2;  and  what  EL3 
thought  about  this  variable  was  compared  to  EL4,  An 
analysis  of  the  results  of  these  comparisons  revealed  that 
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(1)  ELI  and  EL4  encountered  a  net  loss  of  clarity  while 
EL2  and  EL3  reported  a  minimal  gain;   (2)  a  significantly 
lower  degree  of  priority  understanding  appeared  in  EL2 
than  in  ELI.    In  spite  of  those  developments,  none 
constituted  an  effect  of  training.    The  AMA  planning  process 
did  not  clarify  LEA  priorities. 


Item  #6;    Fall  1970  to  Spring  1971 

Item  #6  asks  whether  organizations  feel  The  kinds  of 
things  I  am  doing  will  make  a  long  term  contribution  to 
education  as  a  consequence  nf  a^!A  ♦•raining,    we  can  make  only 
weak  causal  connections  between  training  and  subsequent 
effects  due  to  the  absence  of  Control  LEA's  during  this,  ■ 
and  the  next,  time  period.    Looking  at  the  LEA's  pro-  and 
post-training,  we  found  statistically  significant  change 
in  ELI  and  EL2;  this  change  was  a  downward  movement  in  per- 
ceived importance  of  their  activities.    EL3  and  EL4  also 
minimized  the  importance  of  their  work  but  not  significantly. 
While  all  four  Experimental  LEA's  believed  what  they  were 
doing  to  be  less  worthwhile  after  training  than  before; 
only  ELI  and  EL2's  decline  was  substantial.    We  can  attribute 
training  effects  only  to  ELI  and  EL2,  and  these  effects  were 
negative. 


Item  #6;    Spring  1971  to  Fall  1971 

No  effects  could  be  measured  in  EL3  and  EL4  during 
the  Spring  IS 71  to  Pall  1971  period.    These  Experimental  LEA's 
enhanced  their  felt  importemce  but  not  enough  to  produce 
significant  change.    ELI  and  EL2  showed  sizable  improvement 
in  this  variable  over  time;  the  AMA  program  can  be  credited 
with  a  positive  effect. 


Item  #6;    Fall  1971  to  Spring  1972 

The  addition  of  the  Control  Group  for  each  paired 
Experimental  LEA  makes  our  conclusions  more  valid.    It  also 
provides  multiple  measurements  for  our  findings  of  no 
effects  of  A!1A  training  T3-T4.    None  of  the  Experimental 
groups  differed  from  their  Control  group;  neither  was  ELI 
different  from  EL2  or  EL3  different  from  EL4.  Furthermore, 
none  of  the  LEA's  changed  significantly  between  Fall  1971 
and  Spring  1972.    As  a  result,  no  training  effects  were 
shown  on  this  variable  in  this  period. 


Item  #6;    Fall  1970  to  Spring  1972 

Our  lengthiest  temporal  comparison  merely  reinforces 
this  view.    Between  Pall  1970  and  Spring  1971  the  Experimental 
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lea's  \rere  relatively  stable  in  their  concept  of  how  much 
they  wore  contributing  to  education.    FLl  and  EL^l  declined 
somewhat  while  EL2  and  EL3  each  enhanced  their  positive 
feelings  about  their  work.     In  no  case  however,  were  these 
changes  very  impressive,  one  way  or  the  other,  nor  did  any 
apparent  differences  arise  between  the  States.    Based  on 
this  data,  the  research  team  discerns  no  effects  of  AMA 
training. 


Item  #7;    Fall  1971  to  Spring  1972 

Item  #7,  As  I  see  it,  my  organization  is  moving  in 
the  right  direction,  is  the  last  variable  in  this  section. 
In  general,  the  experimental  LEA's  reacted  to  it  similarly 
over  time;  all  but  one  felt  less  agreement  with  their 
organizational  direction  in  the  Spring  of  1972  than  in  the 
Pall  of  1971.    The  exception,  EL2,  showed  miniscule 
improvement.    Neither  decreases  nor  increases  were  signifi- 
cant.   Bringing  the  Control  groupfs  into  the  analysis  does 
not  modify  this  trend.    In  comparison  with  CLl,  ELI  and 
EL2  did  not  change  greatly  nor  were  they  very  differ- 

ent.   The  same  relationship  and  result  characterizes  EL3 
and  EL4  with  CL2.    It  follows  that  no  training  effects  can 
be  assigned  to  the  ISl'lA  program  in  this  area. 


DATA  ST;r?iiApy 
Development  of  Organizational 
Mission  &  Objectives 

Fall,  1970  -   Spring, 1972 

Type  of  Data 

I  tern  QUE  ST  lONNAI  RE 


IMPACT  OF  TPAINING 


Positive  No 
Effect  Effect 


The  goals  of  this  organiza- 
tion are  articulated  , 

Our  goals  are  realistic  and 
attainable  with  our  best 
efforts . 

The  top  priority  objectives 
for  state  education  are 
clear  to  me* 

I  feel  that  the  objectives 
developed  during  APIA  train- 
ing reflect  the  most  serious 
and  pressing  needs  of  state 
education • 

As  I  see  it,  the  organiza- 
tional priorities  of  the 
objectives  developed  during 
A'^IA  training  are  clear. 


ELI 


Negative 
Effect 


EL2,EL3, 

EL4 


EL1,FL2  EL3,.EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3 , EL4 


241 


6 


The  kindr,  of  things  I  am 
doing  will  make  a  long  term 
contribution  to  education. 


EL1,EL2, 
EL3,EL4 


7 


As  I  see  it,  ray  organization 
is  moving  in  the  right 
direction. 


EL1,EL2, 
EL3,EL4 


Three  instances  in  the  data  indicated  positive  effects 
attributable  to  AMA  training.    ELI  experienced  two  such  posi- 
tive effects,  an  increase  in  the  extent  to  which  people  saw 
goals  as  being  articulated  and  an  increase  in  the  degree 
to  which  the  organization's  goals  were  thought  realistic. 
EL2  also  experienced  an  increase  in  the  realism  attributed 
to  its  goals. 

In  all  other  instances  (25) ,  however,  there  were  no 
significant  effects  (positive  or  negative)  on  the  attitudes 
of  the  personnel  in  the  EL's  as  a  result  of  training.  Atti- 
tudes about  priorities,  objectives  developed  during  A!4A 
training,  and  the  directions  of  their  organizations  remained 
static  over  the  period  of  this  study.    ELS  &  4  also  failed 
to  report  any  clarification  of  their  goals. 

As  a  general  rule,  the  data  indicates  higher  mean 
scores  for  EL's  1  &  2  than  for  EL's  3  &  4. 

The  addition  of  the  Control  LEA's  in  the  T3-T4  compari- 
sons indicates  no  statistically  important  difference  on  any 
of  the  five  items  in  which  they  appear.    The  one  exception 
to  this  is  a  significant  row  variance  in  Item  #1  between  FLl 
and  Control  1.    ELI  was  statistically  different  from  the 
Control;  here  it  had  much  higher  mean  scores  than  those  of 
the  Control  1. 

In  summary,  those  items  indicate  -that  the  AJ!A  training 
seemed  to  have  rio  overall  effect  (with  the  three  exceptions^ 
of  positive  effects  noted)  on  personnel  attitudes  in  the  EL's 
concerning  organizational  Mission  and  Objectives. 


B.    Mobilization  of  Organizational  Planning 

Here  we  assess  attitudes  toward  steps  that  must  be 
taken  to  "gear  up"  for  the  planning  process.    Clarity  of 
organizational  policy  from  which  plans  can  grow,  existence 
of  performance  standards,  assistance  provided  by  a  planning 
unit,  and  evaluation  methods  used  to  measure  the  quality, 
of  what  has  been  done  are  issues  to  be  addressed  in  this 
section.    The  questionnaire  items  used  were: 
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1.  Hy  organization's  policy  statements  are  cloar. 
Points  of  Tine:    Tl,  T?,  T3, 

2.  My  organization's  parformance  standards  are 
understood. 

Points  of  Time:    Tl,  T2,  T3,  T4 

3.  Good  ways  are  used  to  lot  me  know  how  I  can 
improve  my  performance" 

Points  of  Time:    Tl,  T2,  T3,  T4 

4.  I  understand  what  results  must  be  produced  to 
acHievc  the  stated  objectives  of  this  organP" 
zation. 

Points  of  Tine:    T3,  T4 

5.  The  planning  unit  has  been  helpful  to  me. 
Points  of  Time:    T3,  T4 

6.  I  have  good  ways  for  knowing  how  good  our 
results  are. 

Points  of  Time:    Tl,  T2,  T3,  T4 

7.  My  organization  has  reliable  ways  for  knov7ing 
how  well  it  is  attaining  its  obiectives. 

Points  of  Time:     T3,.  T4 

8.  I  think  that  the  objectives  developed  during 
AMA  training  are  clearly  stated  with  regard 
to  results  expected. 

Points  of  Time:    T3,  T4 

lea's:     ELI,  EL2,  EL3 ,  EL^i  .  . 

As  this  certains  specifically  to  AHA  training 
it  was  not  asked  in  the  Control  LEA's  (CLl  and 
CL2) 
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Item  4 


I  understand  what  results  must  be 
produced  to  achieve  the  stated 
objectives  of  this  organization. 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Contr'^1  LEA  #2 
Total 

N 

X 

SD 

N 

X 

SD 

32 

Q.707 

'5.741 

0.855 

?1 

■S,  77J 

1  ,  n?^ 

^? 

1  17r. 

24 

23 

5.043 

1.106 

29 

5.517 

27 

1.618 

.m 

5.566 

l.nnfil 

20 

200 

1.321 

33 

4.fi18 

1.309 

182 

166 

Two  Way  Analysis  of 
Variance 


T3  5  T4 


Experimental 
[Experimental 


LEA#1W/ 
LEA#2 


£SLL 


Row 


Experimental  LEA#3W/ 
Experimental  LEA#4 
Experimental  LEA#1W/ 
pontrol  LEA  #1 


Col 


Row 


Col 


Row 


Experimental  LEA#2W/ 
Control  LEA  #1 

LEA#3W/ 
#2 


Col 


Row 


Experimental 
Control  LEA 


Experimental 
Control  LEA 


LEA#4W/ 
#2 


Row 


Col. 


Row 


0.415 


0.207 


2.227 


0.160 


1.853 
5.366 


1.114 


2.620 


Col.  3.021 


1.385 


0.806 


0.  084 


Signif . 


NS 


NS 


NS 


NS 


NS 
ITT 


NS 


NS 


NS 


NS 


NS 


ERIC 
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Itan     5  The  plannincr  unit  has  been  help- 

ful  to  me. 


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

y" 

A 

ExDeriinen*tal  LEA  #1 

32 

5.  843 

0.954 

31 

5  709 

X  •  u  u  u 

Fym^rimpntAl   T  PA  #9 

ALT  wx  XlMWll  LA X 

ExDerimental  LFA  #3 

31 

1  . 

,  32 

5^218 

1  263 

24 

!• 

22 

5^090 

1.  269 

Experimental  LEA  #4 
Control  LEA  #1 

?9 

1 . 

25 

4,760 

1.535 

30 

5.066 

1.760 

2P 

5.500 

1.357 

Control  LEA  #2 
Total 

36_ 

4.222 

2. 

099 

27 

1  .fin4 

182 

166 

Two  Way  Analysis  of 
Variance 

^3 

F 

Signif. 

Experimental  LEA#1W/ 

Coil 

1,104 

NS 

fexperimental  LEA#2 

Row 

3.977 

.05 

Experimental  LEA#3W/ 

C9l, 

1.039 

NS 

Experimental  LEA#4 

Row 

0.093 

NS 

Experimental  LEA#1W/ 

Col. 

NS 

pontrol  LEA  #1 

,Row 

3.939 

.6S 

Experimental  LEA#2W/ 

Col. 

0.060 

NS 

Control  LEA  #1 

Row 

u.  uy^ 

NS 

Experimental  LEA#3W/ 

Col. 

0.243 

NS 

Control  LEA  #2 

Row 

9.323 

.01 

Experimental  LEA#4W/ 

Col. 

1.528 

NS 

Control  LEA  #2 

Row 

8.285 

.01 
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Item    7         My  organization  has  reliable  ways 
for  knowing  how  well  it  is  attain- 

ing  its  objectives. 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 

N 

X 

SD 

N 

X 

SD 

32 

5.531 

L.077 

31 

5-n64 

0.963 

31 

4.870 

L.231 

32 

4.81? 

1  .n^q 

?4 

5.333 

L.007 

23 

5.nnn 

,_22. 

4-  144 

1  .  541 

.  27 

3.962 

1.55$ 

3n 

4.  5fifi 

1 .430 

4.750 

1  ,  OK") 

4  949 

L82 

166 

Two  Way  Analysis  of 
Variance 


T3  5  T4 


Experimental  LEA#1W/ 
{Experimental  LEA#2 

F 

Sisnif . 

Coll 

1.863 

NS 

Row 

5.621 

.05 

Experimental  LEA#3W/ 
Experimental  LEA#4 

C9li 

1.  870 

NS 

Row 

14.999 

.001 

Experimental  LEA#1W/ 
Control  LEA  #1 

Col. 

0.365 

NS 

,Row 

7.452 

.01 

[Experimental  LEA#2W/ 
Control  LEA  #1 

Col. 

0.065 

NS 

Row 

0.562 

NS 

Experimental  LEA#3W/ 
Control  LEA  #2 

Col. 

1.415 

NS 

Row 

13.392 

.001 

Experimental  LEA#4W/ 
Control  LEA  #2 

Col. 

1.302 

NS 

Row 

0.  548 

NS 
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8   I  think  that  the  objectives  developed  during  AMA 

training  are  clearly  statea  witn  regard  t6  y^SUlts 

expected • 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4  . 

N 

32 

X 

6.187 

SD 
0.820 

N 

31 

X 

5.903 

SD 
1.075 

31 

5.709 

1.442 

32 

5.593 

1.316 

2.4. 

4.916 

2.083 

22 

5.227 

1.546 

21. 

4.793 

1.820 

26 

4.692 

1.805 

116 

111 

Two-Way  Analysis  of 
Variance 

T3    5  T4 

Experimental  LEA#1  W/ 
Experimental  LEA#2 

F 

Signif. 

NS 

Row 

3.467 

NS 

Experimental  LEA#3  W/ 
Experimental  LEA#4 

CqI, 

0.082 

NS 

Row 

0.811 

NS 
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Item  #1:    Fall  1970  to  Spring  1971 

Item  #1,  My  organization's  policy  statements  are 
clear,  Is  the  initial  variable  in  our  analysis  of  the  moblll- 
zation  of  organizational  planning.    Each  of  the  Experimental 
LEA ^s  shows  a  general  downward  trend.    The  degree  to  which 
organization  members  considered  their  policy  statements 
clear  diminished  over  time;  clarity  was  greater  before  train- 
ing than  afterward.    In  ELI  and  EL2  this  reduction  was 
sizable?  in  the  others  not  nearly  so  great.    Thus  the  A'A  can 
be  held  responsible  for  this  negative  training  effect  in  FLl 
and  EL2.    The  lack  of  a  control  group  again  made  judgments 
more  tentative  than  they  might  have  been.    Our  conclusion 
would  have  been  strengthened  had  wc  been  able  to  compare 
changes  in  the  Experimental  LEA's  with  comparable  Control 
LEA'S  which  did  not  receive  any  training.    Yet  the  declines 
in  ELI  and  EL2  are  more  than  adequately  severe  to  warrant 
assignment  of  negative  impact  without  the  addition  of 
Control  groups. 


Item  #1;    Spring  1971  to  Fall  1P71 

The  negative  effects  determined  T1-T2  were  reversed  by 
subsequent  developments  T3-T4,  at  least  in  ELI.    That  organi- 
zation felt  considerably  better  about  their  policy  pronounce- 
ments during  this  period,  and  this  positive  direction  can  be 
credited  to  the  AMA.    No  major  change  v?as  apparent  for  E2 
since  their  increased  average  score  was  negated  by  a  greater 
instability  and  breakdown  of  consensus  (as  evidenced  by  the 
increases  in  the  SD  column) .    Clarity  also  intensified  in 
EL3  and  EL4  to  significant  levels?  and  this  was  attributed 
to  the  AMA  program.    Therefore,  positive  training  effects 
occurred  in  ELI,  EL3,  and  EL4. 


Item  #1;    Fall  1971  to  Spring  1972 

Substantial  differences  opened  betv/con  several  Experi- 
mental lea's  between  Fall  1971  and  Spring  1972.    ELI  believed 
its  statements  more  understandable  than  EL2;  EL3  enjoyed  a 
similar  advantage  over  EL4.    When  we  brought  the  Control 
LEA*s  into  the  analysis,  ELI  and  EL3  maintained  their  superior- 
ity.   EL  surpassed  CLl  and  EL3  bested  CL2  in  perceived  clarity. 
Despite  this,  we  found  it  impossible  to  attach  any  effects 
to  the  training  that  all  received.    The  addition  of  the  Con- 
trol groups  buttressed  our  argument  in  this  regard. 


Item  #1;    Fall  1970  to  Spring  1972 

No  investigation  of  training  effects  would  be  complete 
without  a  direct  comparison  between  pre-  and  post-training 
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statement  clarity.    Such  a  study  was  made  between  the  Fall 
of  1370  and  the  Spring  of  1972  and  revealed  that  training 
had  no  effect  on  any  of  the  Experimental  LEA's.    Each  felt 
similarly  clear  about  their  policy  statements  a  year  after 
training  as  they  had  before  training. 


Item  #2:    Fall  1970  to  Spring  1^71 

Item  #2  is  My  organization's  performance  standards 
are  understood.    Each  or  the  LEA's  who  had  enrolled  in  A^L^ 
training  were  asked  to  respond  to  +:his  statement  before  train- 
ing and  again  immediately  afterward.    Their  answers  recalled 
that  following  their  Hamilton  experience  they  were  more 
confused  about  standards  to  assess  performance.    The  reduced 
compr-^hcnaion  was  relatively  minor  in  EL3,  EL4  but  not  in  ELI 
and  EL2^  where  the  decrease  were  significant.    For  that  reason 
wo  conclude  that  what  happened  in  ELI  and  EL2  v^as  due  to  the 
A?1A  process  and  represents  a  negative  effect  of  training. 


Item  #2!     Spring  1071  to  Fall  1971 

Positive  training  effects  were  reported  during  1S71. 
Both  ELI  and  PL2  recouped  a  sizable  portion  of  their  earlier 
losses.    Their  joint  comeback  was  substantiated  and  can  be 
credited  to  A!^1A.    ELl's  reverse  was  especially  impressive 
since  their  rising  average  score  (as  evidenced  by'' the  X 
column)  was  accompanied  by  a  falling  standard  deviation  (as 
evidenced  by  the  SD  column)  which  meant  leas  diversity  of 
opinion  and  greater  consensus  about  this  largely  favorable 
effect  of  training.    The  same  cannot  be  said  for  EL3  and 
EL4;  no  major  changes  or  differences  occurred  there. 


Item  #2;    Fall  1971  to  Spring  1972 

The  Experimental  LEA's  all  indicated  diminishing 
understanding  of  performance  standards  T3-T4;  although  those 
reductions  viere  not  significant  enough  to  ascribe  any  train- 
ing effect.    Another  measurement  was  available  here  due  to 
the  presence  of  Control  groups  for  each  pair  of  Experimental 
groups.    Their  addition  to  the  evaluation  did  not  change  the 
original  findin^f  of  no  training  effects.    Even  when  matched 
with  an  appropriate  control,  none  of  the  LEA's  indicated 
major  changes.    Our  no  effects  judgment  v/as  thus  reinforced. 


Item  <^2;    Fall  1970  to  Spring  1972 

The  most  powerful  test  of  training  effects  is  to  look 
at  how  well  performance  standards  were  understood  in  the  Pall 
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of  1970  and  then  to  comparG  those  findings  v;ith  those 
gathered  in  the  Spring  of  1972.    Such  a  longitudinal  study 
allowed  us  to  differentiate  between  training  effects  which 
had  some  permanence  and  those  which  were  transitory.  Our 
audit  indicated  negative  training  effects  in  EL2,  EL3  and 
EL4,  and  no  effects  in  ELI.    The  first  three  organizations 
endured  substantial  declines  in  their  understanding  of 
perfomnance  standards,  while  understanding  in  ELI  remained 
stable  over  the  four  periods. 


Item  #3:    Fall  1970  to  Spring  1971 

Item  #3,  Good  ways  are  used  to  let  me  know  hov;  I  can 
improve  my  performance,  was  asked  of  the  four  Experimental 
LEA'S  in  Fall  1970  and  Spring  1971.    Lacking  a  Control  group 
for  comparison  purposes,  we  must  exercise  caution  in  our 
interpretation  but  assignment  of  effects  were  still  possible. 
In  fact,  effects  were  assigned  in  the  cases  of  ELI  and  EL2. 
Both  organizations  felt  substemtially  worse  about  performance- 
related  communication  after  training  than  before,  a  negative 
impact  of  training.    The  other  organizations,  EL3  and  EL4, 
maintained  their  pre-training  opinions  through  the  immediate 
post- training  period. 


Item  j?3t    Spring  1971  to  Fall  1971 

Positive  training  effects  occurred  in  the  LEA's  during 
the  Spring  to  Fall  1971  period.    EL1/EL2  and  EL3/EL4  all 
registered  considerable  increases  in  their  attitudes  toward 
means  of  assisting  performance.    It  follows  that  AMA  can  be 
credited  with  positively  affecting  this  variable. 


Item  #3:    Fall  1971  to  Spring  1972 

Expressed  feelings  about  ways  to  improve  performance 
stabilized  T3-'T4.    This  leveling  off  had  two  bases  of  proof: 
a  comparison  of  one  Experimental  group  with  another  (ELI  and 
EL2)  and  a  comparison  of  Experimental  groups  with  a  Control 
group  (ELI  and  CLl) .    Both  sets  of  calculations  arrived  at 
the  same  conclusion:    no  effects  of  training  in  any  of  the 
four  lea's  v;ho  took  the  team  planning  process  at  Hamilton. 
None  of  them  changed  sufficiently  over  time  to  justify  any 
other  assignment  of  effects. 


Item  n:    Fall  1970  to  Spring  1972 

One  of  the  advantages  of  a  T1-T4  comparison  is  the 
ability  to  see  how  variables  perfojrm  over  the  long  term. 
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Whatever  effects  might  be  registered  in  interim  periods r  EL3 
and  EL4  show  no  effects  of  AKA  training.    EL3  raised  its 
opinions  of  performance  feedback  while  a  slight  decrease 
turned  up  in  EL4.    On  the  other  hand,  ELI  and  EL2  made  sub- 
stantial gains  and  increased  levels  of  consensus  that  good 
means  were  used  to  transmit  performance  data.    For  this  reason 
AMA  can  be  assigned  a  positive  training  effect  in  ELI  and  EL2. 


Item  #4<    Fall  1971  to  Spring  1972 

Item  #4  is  I  understand  what  results  must  be  produced 
to  achieve  the  stated  objectives  of  this  organization.  Com- 
parisons were  made  between  the  Experimental  groups  and  two 
Control  groups.    Weaker  understanding  of  expected  results  was 
shown  in  every  Experimental  LEA  between  Fall  1971  and  Spring 
1972.    In  the  case  of  ELl/CLl,  persons  in  the  Experimental 
group  had  significantly  more  understanding  than  their  Control 
counterparts.    But  none  of  the  Expierimental  LEA's  modified 
their  ccxnprehonsion  significantly  over  time.    Results  were  as 
clear  T4  as  they  had  been  T3.    For  this  reason,  no  training 
effect  could  be  ascribed  to  the  ATIA.  program. 


Item  #5:    Fall  1971  to  Spring  1972 

Item  #5,  The  planning  unit  has  been  helpful  to  me, 
seeks  information  on  contributions  made  by  the  unit  or  indi- 
vidual  assigned  planning  responsibilities  in  the  LEA's.  In 
comparing  LEA's  within  the  same  State  against  each  other 
(ElI  with  EL2  and  EL3  with  EL4)  we  found  that  every  planning 
unit    met  with  declining  appreciation  of  its  work.  These 
declines  were  not  at  significant  levels^  however,  and  no 
training  effects  can  bo  determined.    This  appraisal  is  made 
more  valid  by  comparing  Experimental  with  Control  groups  not 
receiving  training.    In  every  case  where  differences  were 
substantial,  the  Experimental  LEA  was  helped  more  by  its 
planning  unit  than  the  Control  LEA  was  by  theirs.    There  were 
no  significant  changes  over  time  (represented  by  the  Signifi- 
cance column) ,  which  accounts  for  the  absence  of  training 
effects.    Differences  among  the  organizations  were  comparable 
during  both  periods. 


Item  #6:    Fall  1970  to  Spring  1971 

The  evaluation  system  used  to  measure  the  quality  of 
LE^  educational  output  was  the  subject  of  Item  #6,  I  have 
goud  ways  for  knowincy  how  good  our  results  are.    Despite  the- 
fact  that  all  Experimental  LEA' s  thought  their  evaluation 
methods  after  training  were  inferior  to  those  used  before 
training,  their  attitudes  did  not  deteriorate  so  much  as  to 
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make  a  major  pre-  post-training  difference.    In  fact, 
declines  were  so  slight  that  statistical  analysis  considers 
them  insignificant.    No  differences  existed  betv/een  the 
Experimental  groups  during  the  T1-T2  transition.    Am  train- 
ing cannot  be  assigned  any  positive  or  negative  effects  for 
any  of  these  organizations. 


Item  f>6;    Spring  1371  to  Fall  1971 

The  across-the-board  reductions  which  occurred  during 
T1-T2  did  not  reoccur  in  1P71.    All  school  districts  which 
participated  in  the  team  planning  process  at  Hamilton  had 
higher  opinion  of  their  evaluation  methods.    As  far  as  ELI 
and  EL2  are  concerned,  the  rise  was  rather  steep  and  could 
be  the  result  of  AfiA  training.    Increases  reported  in  the 
other  Experimental  LEA*s  were  not  training  effects  since  the 
change  over  time  was  insignificant. 


Item  #6;    Fall  1971  to  Spring  1972 

Here  our  confidence  that  instruments  administered  for 
this  study  are  accurately  measuring  training  effects  is 
appreciably  greater.    For  besides  the  direct  Experimental 
group/Experimental  group  comparisons,  we  also  examined  data 
generated  by  Experimental  group/Control  group  comparisons. 
Both  kinds  of  comparisons  reached  approximately  the  same 
final  judgment:    no  effects  of  training  were  evident  between 
Fall  1971  and  Spring  1972.    Neither  the  Experimental  LEA's 
nor  their  control  counterparts  showed  significant  increases 
or  decreases  to  warrant  attribution  of  training  effects. 


Item  #6:    Fall  1970  to  Spring  1972 

On  an  overall  assessment,  the  AfIA  program  cannot  be 
credited  with  any  training  effects.    Although  ELI,  EL2,  and 
EL3  had  a  more  favorable  opinion  of  their  ways  for  assessing 
how  good  their  results  were  in  the  Spring  of  1?72  than  they 
had  before  training,  in  Fall  1970  this  growth  was  too  incon-, 
sequential  to  come  from  training.    EL4  met  with  less  success; 
persons  in  their  organization  declined  in  their  evaluation 
methods  over  the  18-month  evaluation  program.    Here  again, 
change  was  slight  eind  thus  not  a  training  effect. 


Item  #7:    Fall  1971  to  Spring  1972 

Item,  #7,  My  organization  has  reliable  ways  for  knowing 
how  well  it  is  attaining  its  objectives,  also  attempts  to 
Bieasure  attitudes  toward  the  evaluation  system.    It  was 
applied  to  the  four  Experimental  LEA's  and  two  Control  LEA's 


256 


between  Pall  1971  and  SprJmg  1972.    Analysis  reveals  that 
none  of  the  differences  between  T3-T4  wore  the  result  of 
training.    I^ile  more  reliable  ways  for  evaluating  objective 
attainment  are  present  in  ELI  than  in  EL2,  in  EL3  than  in  EL4, 
in  ELI  than  in  CLl,  and  in  EL3  than  in  CL2,  no  difference  is 
large  enough  to  indicate  corresponding  changes  over  time  in 
any  of  these  organizations.    Changes  between  T3-T4  were  the 
key  test  of  training  effects  since  simple  differences  between 
LEA'S  can  be  the  same  before  and  after  training.  However, 
since  no  significant  changes  occurred,  no  training  effects 
can  be  shown.    The  AMA  program  had  no  measurable  impact  on 
this  aspect  of  local  educational  evaluation. 


Item  #8:    Fall  1971  to  Spring  lv72 

As  a  statement  relevant  only  to  those  organizations 
involved  in  the  AI!A  training  program,  Item  #8,  I  think  that 
the  objectives  developed  during  AMA  training  are  clearTy 
stated  with  regard  to  results  expected,  was  posed  only  to 
the  Experimental  LEA's«    Not  having  an  untrained  Control 
group  with  which  to  compare  results  complicated  analysis. 
What  we  found  was  that  there  were  no  differences  between 
the  Experimental  organizations  or  within  any  organizations 
over  time.    Each  had  the  same  feeling  about  objectivs  clarity 
in  the  Spring  of  1972  as  they  had  in  the  Fall  of  1971. 
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DATA  SUMMARY 

Mobilization  of 
Organizational  Planning 

Fall,  1970  -  Spring,  1972 

Type  of  Data 
Item  QUESTIONNAIRE 


IMPACT  OF  TRAINING 

Positive     Wo  Negative 
Effect    Effect  Effect 


1 

My  organization's  policy 
statements  arc  clear. 

ELl,EL2 
EL3,EL4 

2 

My  organization's  perfor- 
mance standards  are  clear. 

EL2, 

ELI  EL3,EL4 

3 

Good  ways  are  used  to  let 
me  know  how  I  can  improve 
my  performance. 

EL1,EL2  EL3,EL4 

4 

I  understand  what  results 
must  be  produced  to  achieve 
the  stated  objectives  of 
this  organization. 

EL1,EL2, 
EL3,EL4 

5 

The  planning  unit  has  been 
helpful  to  mc. 

EL1,EL2, 
EL3,EL4 

6 

I  have  good  ways  for  knowing 
how  good  our  results  are. 

EL1,EL2, 
EL3,EL4 

7 

My  organization  has  reliable 
ways  for  knowing  how  well  it 
is  achieving  its  objectives. 

BL1,EL2, 
EL3,EL4 

'8 

I  think  that  the  objectives 
developed  during  AMA  train- 
ing are  clearly  stated  with 
regard  to  results  expected. 

EL1,EL2, 
EL3,EL4 

There  were  two  instances  of  positive  and  three  of 
negative  effects  related  to  AtJA  training  in  this  data  about 
the  implCTientation  of  the  planning  process •    In  twenty- 
seven  instances,  no  effects  emerged* 

ELI  and  EL2  both  showed  positive  effects  of  training 
on  Item  There  were  significant  iraproveinents  in  both 

EL's  in  the  amount  oi  feedback  to  staff  from  superiors  or 
supervisors* 


258 


At  the  samo  timo,  negative  training  effects  appear 
in  EL2,  3  and  4  about  performance  standards.    This  is  not 
surprising  as  it  follows  a  pattern  in  the  SED's  and  ref le  .ts 
the  limited  treatment  of  performance  standards  in  the  plans 
produced  during  AMA  training. 

The  rest  of  the  data  in  this  section  follows  the 
pattern  established  in  the  previous  section.    The  majority 
of  items  indicate  no  effects ,  attitudes  remaining  unchanged 
over  the  course  of  this  evaluation. 

A  few  statistically  significant  changes  do  occur 
when  the  EL's  are  compared  with  the  CL's.    In  all  cases / 
however,  these  differences  appear  as  row  vari2tnces  and  indi- 
cate only  that  a  statistical  difference  exists  between  the 
agencies  being  compared.    Although  we  must  carefully  interpret 
all  comparisons  involving  the  Control  LEA*s  because  we  have 
no  pre-'training  data,  the  absence  of  significant  differences 
over  time  between  the  EL's  and  the  CL's  supports  the  overall 
conclusion  that  with  only  two  exceptions,  no  positive  effect 
of  training  is  shown  on  these  items. 


C.    Top  Management  Support  for  Planning 

Crucial  to  the  AMA's  training  design  was  that  top 
management  support  the  planning  process.    The  followirg  six 
items  were  designed  to  obtain  the  perceptions  of  organiza- 
tional participants  on  how  firmly  the  process  had  been  sup- 
ported and  how  that  support  may  have  changed  over  time. 

1.  My  manager  makes  it  clear  that  he  is  committed 
to  the  success  of  our  projects. 

Points  of  Time:    Tl,  T2,  T3,  T4 

2.  My  manager  has  expressed  the  belief  that  the 
AMA^s  training  program  has  been  helpfuTI 

Points  of  Time:    Tl,  T2,  T3,  T4 

LEA^s:     ELI,  EL2,  EL3,  EL4 

As  this  pertains  specifically  to  AMA  training 
it  was  not  asked  in  the  Control  LEA's  (CLl  and 
CL2) 

3.  My  manager  understands  planning  theory  and  is 
e&le  to  put  it  into  practice.  ' 


Points  of  Times    T3,  T4 
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4«    I  bcliovo  my  organization  gives  mo  adoquatc 
training  to  do  my  work  cffGctively, 

Points  of  Time:    T3,  T4 

5.  I  feel  good  about  my  manager's  ability  to  plan. 
Points  of  Time:     T3^  T4 

6.  My  manager  crovides  me  with  adequate  support 
to  pertorm  my  -^ot). 

Points  of  Time:    T3^  T4 


erJc 


I 
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Itan 


My  manager  understands  planning 
theory  and  is  able  to  put  it  into 
practice. 


Fall,  1971 
T3 

Sprirg,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 

N 

X 

SD 

N 

X 

SD 

32 

6.218 

0.792 

_  31 

5.935 

0.8f?3 

"^1 

5.806 

5.750 

1 .077 

24 

5.833 

0.816 

23 

5.391 

1.117 

29 

5.482 

1.1  83 

,.._27 

4.851 

1.725 

30 

5.933 

0.9^0 

_20 

5.550 

1.276 

-^fi.. 

5.22? 

31 

5.161 

1.185 

182 

2  66 

Two  Way  Analysis  of 
Variance 


F 

Sisnif . 

Experimental  LEA#1W/ 
|Experimental  LEA#2 

0.907 

NS 

Row 

2.809 

NS 

Experimental  LEA#3W/ 

C9l. 

4.562 

.05 

[Experimental  LEA#4 

Row 

3.139 

NS 

Experimental  LEA#1W/ 

Col. 

NS 

Control  LEA  #1 

,Row 

5.335 

NS 

Experimental  LEA#2W/ 
Pontrol  LEA  #1 

Col. 

1.028 

NS 

Row 

0.028 

NS 

Experimental  LEA#3W/ 

Col. 

1.093 

NS 

Control  LEA  #2 

Row 

■  3.059 

NS 

Experimental  LEA#4w/ 

Col. 

1.738 

NS 

Control  LEA  #2 

Row 

0.008 

NS 

Item     4        I  believe  mv  organization  gives 

me  adequate  training  to  do  my  work 
effectively. 


Fall,  1971 

Spring,  1972 

T3 

T4 

X 

SD 

N 

A 

ov 

CAp^ximencai  LiCh  ni 

32 

6.093 

0.856 

31 

K  741 

0.  998 

Experimental  LEA  #2 
FiCTi^Ti  mental  LEA  #3 
FynriTT niATit'ji  1   I. FA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 

31 

1  . 

fiRI 

^2 

1.216 

24 

5.625 

0. 

824 

23 

5.521 

0.897 

?9 

4.965 

1.636 

27 

4.851 

1.511 

30 

5.432 

1. 

20 

5.750 

0.966 

36. 

1. 

457 

3? 

5.212 

1.293 

L82 

166 

Two  Way  Analysis  of 
Variance 

^3 

F 

Signif . 

Experimental  LEA#1W/ 
Experimental  LEA#2 

0.294 

NS 

Row 

6.062 

• 

05 

Experimental  LEA#3W/ 
Experimental  LEA#4 

C<?1 

0.177 

NS 

Row 

6.657 

• 

01 

isxperimental  LEA#1W/ 

Col, 

0.008 

NS 

pontrol  LEA  #1 

,Row 

2.877 

NS 

Experimental  LEA#2W/ 
Control  LEA  #3 

Col 

0.736 

NS 

Row 

0.708 

NS 

Experimental  LEA#3W7 

Col 

0.004 

N5 

Control  LEA  #2 

Row 

3.071 

NS 

Experimental  LEA#4W7 

'tol 

0.005 

NS 

Control  LEA  #2 

Row 

1.014 

NS 
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Item     5         I  feel  good  about  my  manager's 
ability  to  plan,  


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

Y 
A 

en 

Experimental  LEA  #1 

6.  417 

fl. 

Tin 

.n 

6.  5qn 

0.863 

Experimental  LEA  #2 

fi.nno 

1.341 

Experimental  LEA  #3 

24 

5.875 

0. 

899 

23 

Experimental  LEA  #4 

?19 

754 

1  . 

27 

S  444 

1  ^^4fl 

Control  LEA  #1 

3n 

I' 

066 

n  71 

Control  LEA  #2 

5.R05 

1.450 

fi.  nnn 

0.935 

Total 

182 

166 

Two  Way  Analysis  of 
Variance 

^3 

1 

^4 

F 

Signif . 

Experimental  LEA#1W/ 

Col. 

n.4S>5 

,  N5 

Experimental  LEA#2 

Row 

4.949 

Experimental  LEA#3W/ 

,C9l, 

NS 

Experimental  LHAt#4 

Row 

0.010 

NS 

Experimental  LEA#1W/ 

Col. 

0.638 

.  NS 

pontrol  LEA  #1 

,Row 

0.113 

NS 

Experimental  LEA#2W/ 

Col. 

0.242 

NS 

Control  LEA  #1 

Row 

2.766 

NS 

lExperimental  LE^#3W/ 

Col. 

0.542 

NS 

Control  LEA  #2 

Row 

1.665 

NS 

Experimental  LEA#4W/|Col. 

0.030 

NS 

Control  LEA  #2  | 

KOW 

1.721 

NS 

1 
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Item     6       My  manager  provides  me  with  adequate 
support  to  perform  my  job.  


Fall,  1971 

Spring,  1972 

T3 

1 4 

N 

X 

SD 

N 

X 

SD 

Experimental  LEA  #1 

?2 

6.500 

0. 

508 

31 

6.419 

0.672 

Experimental  LEA  #2 
Experimental  LEA  #3 

21 

1  .395 

32 

5.843 

1.66(5 

24 

0. 

974 

23 

5.782 

1.241 

Experimental  LEA  #4 
Control  LEA  #1 

.S.793 

1.206 

,27 

5.185 

1.545 

30 

5.966 

I, 

351. 

20 

6.050 

0.998 

Control  LEA  #2 
Total 

Ah 

1  . 

457 

33 

5.787 

1  TOO 

182 

166 

Two  Way  Analysis  of 
Variance 

^4 

F 

Signif. 

experimental  LEA#1W/ 

0,023 

NS 

jExperimental  LEA#2 

Row 

15.428 

.001 

Experimental  LEA#3W/ 

C9l 

2.193 

NS 

Experimental  LEA#4 

Row 

2.071 

NS 

Experimental  LEA#1W/ 

Col. 

0.000 

NS 

pontrol  LEA  #1 

■  Row 

fi.471 

.05 

Experimental  LEA#2W/ 

Col. 

n.2in 

NS 

Control*  LEA  #1 

Row 

0.956 

NS 

Experimental  LEA#3W/ 

Col. 

0.195 

NS 

Control  LEA  ^2 

Row 

0.011 

NS 

Experimental  LEAff4W/ 

Col. 

1.981 

NS 

Control  LEA  #2 

Row 

1.920 

NS 
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Item  n:    Fall  1970  to  Spring  1971 

Item  #1/  My  manager  makes  it  clear  that  he  is  com- 
mitted to  the  success  of  out  projects  ^  is  the  first  variable 
to  be  considered  in  our  study  of  Top  Management  Support 
for  Planning.    None  of  the  Experimental  LEA's  differed 
significantly  from  each  other,  either  before  or  immediately 
after  training.    But  regarding  change  in  managerial  commit- 
ment within  each  organization,  a  considerable  reduction  was 
noted  in  ELI.    From  showing  the  highest  level  of  commitment 
before  training,  ELI  dropped  to  the  lowest  inuiiediately  after 
training.    A  fall  of  such  magnitude  was  ascribed  to  a  nega- 
tive effect  of  training.    EL2,  EL3,  and  EL4  also  dropped  in 
this  regard;  however,  they  did  not  decline  nearly  so  much 
as  ELI,  nor  could  we  measure  any  influence  of  the  process  at 
Heunilton. 

Item  f^l:    Spring  1971  to  Fall  1971 

In  every  Experimental  group,  managers  became  much 
more  committed  to  project  success  between  Spring  and  Fall  of 
1971.    Major  increases  on  this  variable  produced  a  positive 
training  effect  in  ELI,  EL2,  EL3,  and  EL4.    The  most  impressive 
comeback  occurred  in  ELI;  this  observation  has  <?lmost  appeared 
standard  operating  procedure  for  that  group  in  . 2-T3  compari- 
sons.   They  were  highest  in  Tl;  dropped  to  lowest  in  T2; 
and  now  have  climbed  back  to  the  summit  of  the  list.  This 
is  particularly  surprising  considering  that  ELI  was  able  to 
regain  its  Tl  position  at  a  time  v/hen  their  "competition" 
was    also  registering  sizable  increases  in  expressed  managerial 
support . 

Item  #1;    Fall  1971  to  Spring  1972 

The  upward  trends  of  the  preceding  period  continued 
for  ELI;  they  also  increased  their  lead  over  most  of  the 
field.    By  contrast  to  EL2  and  CLl,  they  had  a  significant 
advantage  in  managerial  support  for  projects.    There  were 
no  other  changes  or  differences,  and  thr.  rise  observed  in  LLl 
was  insignificant.    Thus,  even  considering  the  Control  groups^ 
about  the  same  commitment  v»as  recorded  in  Fall  1971  as  in 
Spring  1972.    Under  these  circumstances,  no  training  effects 
can  be  assigned. 

Item  ni    Fall  1970  to  Spring  1972 

An  overall  evaluation  of  what  the  WIA  program 
accomplished  must  compare  the  observed  support  for  projects 
before  training  with  current  levels  of  support.    Such  a  Pall 
1970  to  Spring  1972  CMiparison  is  the  best  basis  on  which 
to  assess  the  influence  of  AI1A  change  strategies.  The 
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evalu^.tion  reveals  that  no  training  effects  existed  in  any 
of  the  organizations;  none  of  then  changed  over  time  to  a 
degree  sufficient  to  justify  any  causal  connection  of  change 
to  the  AMA  training  experience. 

Item  #2:    Fall  1070  to  Spring  1971 

*  Iten  #2^  My  manager  has  expressed  the  belief  that  the 
AMA' s  training  program  has  been  helpful ^  pertains  only  to 
Experimental  LEA's*    Comparing  statements  by  managers  in 
these  organizations  before  and  after  they  went  up  to  Hamilton 
provides  some  evidence  of  training  effects.    EL3  and  EL4 
experienced  positive  effects  from  being  trained  in  planning 
by  AI4A;  both  shoii^ed  gains  in  favorable  attitudes  expressed 
by  superiors  toward  the  AMA  program.    Since  some  occurrence 
during f  or  closely  connected  with,  the  training  process  must 
have  made  them  feel  this  way,  positive  effects  of  training 
could  be  attributed  to  EL3  and  EL4.    Managerial  support  for 
the  training  program,  on  the  other  hand,  was  stable  in  ELI 
and  EL2. 

Item  #2t    Spring  1971  to  Fall  1971 

In  this  period  neither  EL3  nor  EL4  vaxiod  their 
opinions  about  training,  nor  did  any  significant  differences 
appear  between  them.    In  ELI  and  EL2,  substantial  growth 
in  favorable  attitudes  was    measured.    This  increase  was 
sufficient  to  claim  positive  training  effects  for  both  organi- 
zations.   No  similar  conclusion  for  ELS  and  EL4  was  possible. 


Item  #2:    Fall  1971  to  Spring  1972 

All  of  the  Experimental  LEA's  found  their  managers 
less  supportive  of  the  AMA  program  in  Spring  1972  than  they 
had  been  in  Pall  1971.    No  effects  of  training  could  be 
assigned,  however,  due  to  the  insignificance  of  this  decline. 
SLl  was  substantially  more  favorable  toward  training  than 
EL2  but  neither  changed  over  time. 

Item  #2;    Fall  1970  to  Spring  1972 

The  AMA  was  credited  with  positive  training  effects 
in  EL3  and  EL4  for  t^is  period.    In  both,  organization 
managers  thought  the  AMA  program  was  more  helpful  a  year 
after  training  had  eiided  than  they  had  expected  it  to  be 
beforehand.    No  training  effects  occurred  in  SLl  and  EL2 
although  ELI  managers  were  much  more  favorably  disposed 
toward  the  AMA  program  than  their  counterparts  in  EL2. 
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Item  <t3;    Fall  1971  to  Spring  1972 

Item  #3  is  My  manager  understands  planning  theory  and 
is  able  to  put  it  into  practice^    The  four  Experimental  LEA's 
and  the  Control  LEA's  were  asked  to  respond  to  this  state- 
ment.   Subsequent  analysis  indicated  no  training  effects 
except  in  the  cases  of  SL3  and  EL4.    Both  organizations 
thought  less  of  managerial  competence  in  planning  in  the 
Spring  than  they  had  during  the  preceding  Fall,  a  decline 
that  constituted  a  negative  training  effect.    No  other  compari- 
son revealed  differences  between  the  groups  or  within  one 
group  over  time. 

Item  #4;    Fall  1971  to  Spring  1972 

Item  #4,  I  believe  my  organization  gives  me  adequate 
training  to  do  my  work  effectively/  was  applicable  bctv^een 
the  Fall  of  1971  and  the  Spring  of  1972.    The  status  of  in- 
service  training  was  higher  in  ELI  than  in  EL2/  while  EL3 
thought  more  highly  of  their  in-service  program  than  EL4  did. 
Despite  these  differences  between  Experimental  LEA's,  no 
difference  was  considerable  enough  to  create  a  change  over 
time,  i.e.,  disparities  in  the  Fall  continued  into  the  Spring 
at  nearly  the  s^^e  relative  levels.    Addition  of  the  control 
groups  did  not  alter  this  impression;  Experimental/Control 
comparisons  revealed  no  differences  or  changes  in  the  quality 
of  in-service  training  made  available  to  members  of  those 
organizations.    On  account  of  this  prevailing  similarity, 
the  TiHA  was  not  responsible  for  any  training  effects  in 
trainee  LEA's. 

Item  #5:    Fall  1971  to  Spring  1972 

The  effect  of  AMA  training  on  managerial  support  for 
planning  was  also  measured  by  an  analysis  of  reactions  to 
Item  #5,  I  feel  good  about  my  manager's  ability  to  plan. 
All  organizations  encountered  minor  decreases  xn  this  area 
between  Fall  1971  and  Spring  1972.    But  in  T3,  ELI  felt 
best  about  superiors'  competence  in  planning  and  this  high 
position  carried  into  T4.    SL2  also  showed  much  less  co^^i-^ 
donee  in  this  managerial  skill  than  ELI  had  shown  in  tht^ir  - 
managers.    Aside  from  these  secondary  considerations/  no 
difference  existed  between  EL3/EL4  and  none  cz  the  Experi- 
mental LEA's  modified  their  opinions  over  time.    The  absence 
of  change  was  not  reversed  by  evaluation  of  the  Control 
LEA's.    Comparisons  between  the  Experimental  LEA's,  which 
received  training,  and  the  Control  LEA's,  which  did  not, 
reveals  no  differences  or  changes.    It  follows  frcwi  this 
evidence  that  the  AI4A  cannot  be  responsible  for  any  train- 
ing effects. 
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Item  #6;    Fall  1971  to  Spring  1972 

Item  #6,  My  manager  provides  mi  with  adecniate  support 
to  perform  my  job]  is  the  final  variable  in  this  section. 
Except  for  EL2,  the  Experimental  LEr*'s  all  showed  slight 
decreases  between  T3-T4  in  support  given  by  superiors  to 
their  subordinates'  jobs.    The  small  loss  in  EL2,  however, 
was  not  enough  to  avoid  being  significantly  lower  than  the 
support  levels  measured  in  ELI.    ELI  was  also  much  higher 
than  the  Control  LEA  in  this  regard.    No  other  differences 
or  changes  of  significance  were  reported.    The  fact  that  both 
Experimental  and  Control  comparisons  revealed  no  movement 
either  way  gives  us  more  confidence  in  our  measurements. 
Consequently,  no  training  effects  were  registered  in  any  of 
the  Experimental  LEA's  concerning  job-related  assistance  to 
subordinates. 


DATA  SUIIMARY 

Top  Manigement  Support 
for  Planning 

Fall,  1970  -  Spring.  1972  IMPACT  OF  TRAINING 


Type  of  Data  Positive       No  Negative 

Item  QUESTIONNAIRE  Effect      Effect  Effect 


1 

My  manager  n.akes  it  clear  he 
is  coiranii-.ted  to  the  success 
of  our  projects. 

EL1,EL2, 
EL3,EL4 

2 

My  manager  has  expressed  the 

belief  that  the  AMA's  train-  EL3,EL4 

ing  program  has  been  helpful. 

EL1,£L2 

3 

rly  manager  understands  plan- 
ning theojry  and  is  able  to 
put  it  into  practice. 

EL1,EL2  EL3,EL4 

4 

I  believe  my  organization 
gives  mo  adequate  training 
to  do  ray  work  effectively. 

EL1,EL2, 
EL3,EL4 

5 

I  feel  good  about  my  manager's 
ability  to  plan. 

EL1,£L2, 
EL3,EL4 

6 

My  manager  provides  me  with 
adequate  support  to  perform 
my  job. 

EL1,EL2, 
EL3,EL4 

ERIC 
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This  category  of  twenty-four  variables  shows,  with 
four  exceptions  (two  positive  and  two  mgative),  no  si^nif i- 
£^  ffffg^g  QJL  training  on  top  managenent  support  for  plan- 
nxng  in  the  Local  Education  Agencies. 

Two  positive  effects  were  recorded  in  EL3  and  EL4 
concerning  the  support  of  top  management  for  the  A?1A  program. 
These  same  school  districts  experienced  statistically 
significant  drops  in  their  mean  scores  in  a  T3-T4  comparison 
concerning  their  managers*  knowledge  of  planning  theory. 
Apparently  there  was  increased  av^areness  and  consequently 
increased  frustration  as  a  result. 

Again,  all  the  other  items  in  this  category  reflect 
the  consistent  pattern  in  the  EL's  of  no  training  effects, 
at  least  for  raist  items. 

The  addition  of  the  CL's  in  T3-T4  produced  only  two 
significant  Row  variances  in  this  data,  indicating  that 
generally  no  differences  appeared  between  the  Experimental 
el's  and  the  CL's. 

The  reader  is  again  reminded  that  this  data — like  all 
the  attitudinal  data — must  be  interpreted  with  caution 
because  we  lacked  a  pro-test  Control  group.    The  same  admoni- 
tion applies  to  the  data  that  appears  in  T3-T4  only. 


n     Credibility  of  thc^  Planning  Process 

The  data  below  refer  to  the  assumption  that,  in  order 
to  be  effective  within  the  organization,  the  planning  pro- 
cess must  be  thought  credible  by  it.    We  are  interested  in 
how  important  the  LEA's  consider  planning,  and  what  role 
the  training  program  played  in  redefining  that  role.  Pour 
questionnaire  items  address  this  issue. 

1*    As  I  see  it,  planning  is  an  integral  part  of 
running  the  state's  schools^ 

Points  of  Time:     Tl,  T2,  T3,  T4 

2 •    As  I  see  it/  persons  in  this  organization  put  a 
lot  of  effort  into  planning. 

Points  of  Time:    T3,  T4 

3.    My  capability  to  plan  effectively  will  positively 
affect  my  future  career  in  this  organization. 

Points  of  Time:     T3,  T4 

^*    The  activities  relating  to  planning  are  having  an 
effect  on  the  policy  or  this  organization. 


Points  c    Time:    T3,  T4 


r 
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2         As  I  see  it,  persons  in  this 

organization  put  a  lot  of  effort 
into  planning. 


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

V 

X 

O!/ 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3' 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 

5.812 

0. 

820 

32 

o .  o  uu 

J.  .  J.  «3  3 

44fi 

32 

n  Qi  ^ 

0  A 

6.000 

0. 

780 

23 

*      £.  X 

1  122 

9Q 

5.27S 

1. 

360 

27 

4.370 

1.390 

5.400 

1. 

la^ 

1   2Q  ■ 

S  050 

1.  276 

Ifi 

1  . 

1  .241 

L82 

166 

Two  Way  Analysis  of 
Variance 

^3 

u 

■'4 

F 

Sienif. 

Experimental  LEA#1W/ 
fexperimental  LEA#2 

0.9fi7 

MS 

Row 

3.571 

NS 

Experimental  LEA#3W/ 
F  '.perimental  LEA#4 

C9l 

8.412 

.01 

Row 

15.  45 

4 

.001 

Experimental  LEA#1W/ 
pontrol  LEA  #1 

Col 

2.565 

NS 

,Row 

4.348 

.05 

Experimental  LEA#2W/ 
Control  LEA  #1 

Col 

0.  844 

NS 

Row 

0.071 

NS 

Experimental  LEA#3W/ 
Control  LEA  #2 

Col 

0.917 

NS 

Row 

4.257 

.05 

Experimental  LEA#4W/ 

Col 

3.171 

NS 

Control  LEA  #2 

Row 

4.087 

.05 
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It«n  .J   My  capability  to  plan  effectively 

will  positively  affect  my  future 
career  in  this  organization. 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Fall,  1971 
T3 

Spring,  1972 
T4 

N 

X 

SD 

N 

X 

SD 

.32 

5.9fifi 

1.402 

-i- 

5.870 

1.258 

31 

0.820 

5.750 

1.01$  . 

24 

5.958 

1.398 

23 

5.782 

1.277 

.23 

0.fi4fl 

27 

5.888 

1.250 

30 

^.733 

I.4fi0 

20 

5.900 

1.209 

Jl£- 

1.245 

.  3? 

5.969 

182 

166 

Two  Way  Analysis  of 
Variance 


T3  §  T4 


Experimental  LEkUVH/ 
texperimental  LEA#2 

F 

Sijjnif . 

Col  1 

NS 

Row 

0.030 

NS 

lExperimental  LEA#3W/ 
Experimental  LEA#4 

.C9l, 

1.500 

NS 

Row 

0.8517 

NS 

Experimental  LEA#1W/ 
pontrol  LEA  #1 

Col. 

0.017 

NS 

.Row 

0.159 

NS 

Experimental  LEA#2W/ 
Control  LEA  #1 

Col. 

0.313 

NS 

Row 

0.404 

NS 

Experimental  LEA#3W/ 
Control  LEA  #2 

Col. 

0.566 

NS 

Row 

0.643 

NS 

Experimental  LEA#4W/ 
Control  LEA  #2 

Col. 

2.127 

NS 

Row 

0.021 

NS 
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Itm    4         The  activities  relating  to  planning 
are  having  an  effect  on  the  policy 
of  this  organization.  " 


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

X 

SD 

Experimental  LEA  nl 

jZ 

6.218 

0.750 

31 

J  •  0  /  u 

Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  nl 
Control  LbA  n/, 
10  tai 

J  X 

5.387 

1.453 

32 

X  •  X  «/  / 

OA 

5.625 

0.923 

23 

D  •  X  «3  U 

X  •  u  «^  / 

5Q 

1-250 

26 

*z  •  7  ^  <.J 

1  547 

1.048 

20 

0.994 

?fi 

A   Q7  9 

1  .fifi4 

71 

1  .•^44 

182 

166 

Two  Way  Analysis  of 
Variance 

F 

Sienif . 

Experimental  LEA#1W/ 
Experimental  LEA#2 

Col 

0.841 

NS 

Row 

11.453 

.001 

Experimental  LEA#3W/ 
Experimental  LEA#4 

Q9I 

7.037 

NS 

Row 

0.049 

NS 

Experimental  LEA#1W/ 

Col 

1.835 

MS 

Control  LEA  #1 

Row 

4.536 

.05 

[Experimental  LEA#2W/ 
Control  LEA  #1 

Col 

0.100 

NS 

Row 

1.557 

NS 

Experimental  LEA#3W/ 

Col 

1.5?q 

NS 

Control  LEA  #2 

Row 

3.459 

NS 

Experimental  LEA#4W/ 

Col 

3.041 

NS 

iControl  LEA  #2 

Row 

2.435 

NS 
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Item  #1;    Fall  1D70  to  Spring  1971 

Item  #1  of  this  section,  As  I  see  it,  planning  is  an 
integral  part  of  running  the  State's  schools/  was  put  to 
persons  in  the  Experimental  LEJl's  across  four  points  in  time 
and  to  people  in  Control  LEA's,  twice.    Planning  was  seen 
as  important  by  all  four  Experimental  groups  between  Pall 
1970  (prior  to  training)  and  Spring  1971  (immediately  after 
training) .    No  great  differences  between  any  of  these 
organizations  were  noted.    However,  ELl's  estimate  of  plan- 
ning steeply  declined.    Their  reductions  were  significant, 
and  were  neld  to  be  negative  training  effects.    None  of  the 
other  groups  seemed  to  be  affected  one  way  or  the  other. 


Item  #lt    Spring  1971  to  Fall  1971 

During  1971,  ELI  attached  much  more  importemce  to 
planning  than  earlier;  their  opinion  of  it  rose  so  far  that 
positive  training  effects  could  be  assigned.    No  other 
Experimental  group  experienced  a  compareJale  increase,  although 
all  rose  somewhat.    Substantial  differences  did  not  exist 
among  any  of  them. 

Item  #1;    Fall  1971  to  Spring  1972 

The  AMA  is  not  accountable  for  any  training  effects 
during  this  period.    This  decision  is  based  not  only  on  our 
usual  Experimental  group  comparisons,  but  also  on  comparisons 
of  each  organization  which  had  training  to  other  Control 
organizations  which  did  not  undergo  training.    Increases  or 
decreases  in  how  important  planning  was  thought  by  the 
Experimental  LEA's  was  thus  contrasted  to  change  in  the 
Control  lea's.    The  Experimental  group  and  Experimental/Con- 
trol group  comparisons  reached  identical  conclusions:  no 
training  effects.    None  of  the  LEA's  changed  their  views 
between  Pall  1971  and  Spring  1972,  nor  did  any  differences 
appear  between  them. 

Item  #1:    Fall  1970  to  Spring  1972 

To  assess  the  overall  impact  of  AMA  training  on  ex- 
pressed LEA  attitudes  toweird  the  integral  role  of  planning 
in  State  school  administration,  a  longitudinal  comparison 
was  made.    Each  Experimental  LEA's  feeling  on  this  issue 
before  training  was  matched  with  its  attitudes  a  year  after 
training.    In  no  case  did  an  LEA  differ  substantially  from 
another  or  from  itself  over  time.    Thus  we  ascribed  no 
effects  of  AMA  training  to  any  Experimental  LEA. 
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Item  #2:    Fall  1971  to  Spring  1972 

Item  #2  is,  As  I  see  it,  persons  in  this  organization 
put  a  lot  of  effort  into  planning.    It  attempted  to  evaluate 
effort  invested  in  the  planning  process  by  the  Experimental 
LEA'S  between  the  Fall  of  1971  and  the  Spring  of  1D72.  Addi- 
tion of  Control  lea's  buttressed  and  confirmed  conclusions 
made  on  the  basis  of  the  Experimental  LEA  comparisons  alone. 
In  reference  to  the  Control  LEA's,  both  ELI  and  EL3  put 
more  work  into  planning  than  CLl  and  CL2  respectively;  and 
EL4  put  les3  than  either  Control  group.    The  gap  between  EL3 
and  EL4  on  this  issue  and  the  major  reduction  of  planning- 
related  effort  in  EL4  over  time  warrant  assignment  of  nega- 
tive effects  in  that  organization,    EL4  did  less  work  associ- 
ated with  planning  in  T4  than  they  had  in  T3. 

Item  #3;    Fall  1971  to  Spring  1972 

One  worthwhile  evaluation  which  can  be  made  in  order 
to  study  the  credibility  of  the  planning  process  is  to  see 
how    graat ,  a  contribution  expertise  in  planning  made  to 
advancement  within  the  Experimental  LEA's,    If  the  connection 
was  close  and  persons  in  those  organizations  felt  planning 
could  aid  their  careers,  this  could  be  a  good  indication  of 
the  future  success  of  the  planning  process.    All  LEA's  had 
declining  scores  on  My  capability  to  plan  effectively  will  posi- 
tively affect  my  future  career  in  the  organization  (Item  j?3) 
between  Fall  1971  and  Spring  1972.    But  in  no  case  was  the 
decrease  significant  nor  was  there  a  difference  between  any 
of  the  Experimental  or  Control  groups.    Statistically  speak- 
ing, planning  skills  rnd  future  careers  had  comparable  con- 
nections made  between  them  during  both  points  of  time. 
Consequently,  it  was  not  possible  to  attribute  any  effects 
of  training  to  the  AMA  program. 

Item  #4;    Fall  1971  to  Spring  1972 

The  connection  between  planning  and  policy  was  made 
on  Item  #4,  The  activities  relating,  to  planning^are  having  an 
effect  on  the  policy  of  this  organization.    Differences  were 
observed  between  ELI  and  EL2  and  CLl.    The  influence  of  plan- 
ning in  that  case  was  greater  in  ELI  than  in  the  other 
Experimental  or  Control  groups.    No  other  sizable  differences 
existed.    In  all  LEA^s,  planning's  connection  with  policy 
was  viewed  as  getting  more  tenuous  as  time  passed;  however, 
the  lessened  influence  was  too  small  to  hold  the  AMA  train- 
ing program  responsible  for  it.    For  that  reason,  no  training 
effects  were  determined  in  any  Experimental  LEA. 
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DATA  SUMMARY 

Planning  Process 
Pall,  1970  -  Sorinq,  1972 

Type  of  Data 
Item  QUESTIONNAIRE 

IMPACT  OP  TRAINING 

Positive  No 
Effect  Effect 

Negative 
Effect 

1     As  I  see  it,  planning  is  an 
integral  part  of  running  the 
state's  schools. 

EL1,EL2, 
EL3,EL4 

2     As  I  see  it,  persons  in  this 
organization  put  a  lot  of 
effort  into  planning. 

EL1,EL2, 
EL3 

EL4 

3     My  capability  to  plan  effec- 
tively will  positively 
affect  my  future  career  in 
this  organization. 

EL1,EL2, 
EL3,EL4 

4     The  activities  relating  to 
planning  are  having  an  effect 
on  the  policy  of  this  organi- 
zation. 

EL1,EL2, 
EL3,EL4 

With  remarkable  consistency,  the  credibility  of  the 
planning  process  in  the  Experimental  LEA's  was  not  affected 
by  AMA  training.    These  organizations  viewed  planning  in 
nearly  the  same  light  both  before  and  after  training.  Thd 
only  exceptions  to  this  general  rule  were  Items  #2  and  #5., 

The  effort  devoted  to  planning  (item  #2)  was  unaffected 
by  training  in  ELI,  EL2,  or  EL3.    However,  in  EL4,  it  was 
generally  felt  that  people  did  not  spend  as  much  time  on 
planning  in  Spring  1972  as  they  had  done  in  Pall  1971.  The 
other  Experimental  LEA's  also  experienced  declining  levels 
Of  planning  input;  but  only  in  the  case  of  EL4  was  this 
reduction  sizable  enough  to  justify  attributing  negative 
effects  of  training  to  AMA. 

This  decision  on  effect  was  made  only  after  careful 
consideration  dictated  by  the  absence  of  pr^^-training  data 
from  Tl.    Interpretation  of  T3-T4  data  was  slightly  compli- 
cated because  no  baseline  data  existed  against  which  to 
oomoare  recent  changes  in  order  to  determine  training  effects. 
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Control  LEA'S  were  not  inserted  in  the 
study  until  T3-T4,  tho  usual  T1-T4  test  for  training  effects 

no?  S^^^?^^^?^^:  effects  of  training  could 

not  be  determined  using  an  Experimental  group/Control  group 
comparison  between  Pall  1970  and  Spring  1972.    For  an 
acceptable  substitute,  the  research  team  retained  the  T1-T4 
time  frames,  but  compared  Experimental  LEA's  within  the 
same  state  (ELI  with  EL2,  and  EL3  with  EL4). 

AMn  i.    ^°^^^^^^^^9  this  evidence,  we  concluded  overall  that 
training  did  not  influence  the  credibility  of  the  plan- 
ning process  IrTtKe^Experimental  LEA's. 


AREA  4;     SUMMARY  AMD  CONCiLUSIONS 

In  this  chapter  we  have  discussed  the  actions  taken 
by  the  Experimental  organizations  toward  meeting  the  first 
thirteen  training  goals  and  criteria  of  tho  AHA,  as  well 
2f,.«  Sk^o  °^  organizational  participants  toward  them. 

Here  this  massive  collection  of  data  is  summarized  to  give 
readers  a  clear  overview  of  what  follows. 

Linking  Program  with  Organizational  Impact 

This  evaluation  effort  is  base*d  on  the  premise  that 
a  direct  relationship  exists  between  the  design  of  the 
training  program,  the  way  in  which  it  was  conducted  and 
potential  organizational  impact. 

In  our  previous  discussion  of  the  training  orograra^^ 
we  mentioned  three  types  of  attitude  change—identification, 
compliance,  and  internalization.    The  mh  training  program 
was  defined  as  one  which  primarily  required  identification 
and  compliance  on  the  part  of  its  clients. 

This  conclusion  was  based  on  an  analysis  of  training 
design  and  its  mode  of  implementation.    At  all  points  of 
the  planning  process,  the  boundaries  of  legitimate  discourse 
and  program  content  were  controlled  by  the  AMA  trainer. 
These  boundaries  were  based  on  the  AMA's  concept  of  effec- 
tive organizational  planning.    The  main  emphasis  of  its 
program  was  to  teach  a  planning  process;  to  this  end,  dis- 
cussion within  the  group  was  held  to  the  level  of  rational 
dialogue  and  exchange  of  opinion. 

This  analysis  concluded  that  in  order  for  attitude 
change  to  occur  within  the  context  of  a  program  of  this 

43 

Chapter  One  of  this  report  and  the  Introduction  to 
Chapter  Pour. 
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nature,  the  individual  must  identify  with  the  other  members 
of  his  team  and  comply,  that  is,  do  what  v/as  expected  of 
him  in  a  particular  situation.    An  external  stimulus  would 
be  necessary  to  produce  a  change  in  attitude. 

On  the  other  hand,  internalized  attitude  change  is 
incorporated  into  the  person's  own  values  and  does  not 
depend  on  external  support  for  activation.    In  order  for 
internalized  attitude  change  to  occur,  however,  both  logical 
discourse  and  emotions  must  be  considered  in  developing  a 
training  program.    As  has  been  discussed  above,  this  was 
not  the  case  in  the  A!1A  program.    Since  most  program  time 
was  allotted  to  lectures,  the  AMA's  primary  vehicle  of 
attitudinal  change  was  the  trainees'  compliance  and  identifi- 
cation with  the  concepts,  experiences,  and  values  of  the 
lecturers . 

Such  attitude  change  is  usually  evanescent  unless 
it  receives  continual  reinforcement.    It  may  also  result  in 
pseudo-compliance  on  the  part  of  trainees. 

Because  we  consider  attitude  change  a  necessary 
though  not  a  sufficient  condition  of  altering  organizational 
behavior  and  improving  organizational  effectiveness,  we 
have  presented  in  this  chapter  evidences  of  both  attitudes 
and  actions.    One  without  the  other  is  meaningless,  we  think. 

A  summary  of  the  attitudes  and  actual  progress  tov;ard 
the  first  thirteen  PiMh  training  goals  is  presented  below, 
followed  by  conclusions  based  on  the  data. 
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SUMTIARY  OF  ACTION  FINDINGS 
SED  LEVEL 
MiA  CRITERI?x  1-13 
PALL,  1970  to  SPRING,  1972 

A!40UNT  OP  PROGRESS 


No>  Criteria  Minimum   Moderate  Maximum 

1.  Agreed  upon  a  definition  of 

the  institution's  mission  E1,E2 

2.  Established  continuing  objec- 
tives and  planning  procedures 
for  long-range  achievement  of 

the  institution's  mission  Bl,E2 

3.  Identified  resources  and  con- 
straints E1,E2 

4*  Differentiated  between  where 
the  institution  is  going  and 
where  it  wants  to  go  E1,E2 

5*    Modified  previously  estab- 
lished objectives  r^l,E2 

6.  Identified  and  analyzed  alter- 
native courses  of  action  E1,E2 

7.  Determined  priorities  El  E2 

8*    Made  strategic  action  as-  El  E2 

signments 

9.    Defined  standards  of  performance 

for  key  administrators  El  E2 

10.    Specified  task  completion  dates  El  E2 

11*    Designed  supplementary  plan- 
ning efforts  E1,E2 

12.    Assigned  responsibilities  to 

subordinate  units  E1,E2 

13*    Designed  a  methodology  by 

which  future  performance  may 

be  evaluated  in  relation  to 

the  performances  specified  in 

the  plan  E1,E2 
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SOMMARY  OF  ATTITUDINAL  FINDINGS 

DATA  SUMMARY 

CAUSAL  VARIABLES  -  STATE 
DEPARTMENT  OF  EDUCATION 

Fall,  1970    -    Spring,  1972 


IMPACT  OF  TRAINING 


Positive       No  Negative 
Item  Type  of  Data  Effect     Effect  Effect 

DEFINITION  OF  THE  MISSION  OF  THE  ORGANIZATION 
CONTENT 

1  Definition  of  the  Institution's 

Mission  El  E2 

2  Sense  of  SED  Mission  E1,E2 

3  Feelings  about  the  direction  the 
organization  is  moving  El,E2 

QUESTIONNAIRE 

4  The  kinds  of  things  I  am  doing 
will  make  a  long  term  contribu- 

tion  to  education.  El  E2 

DEVELOPMENT  OF  ORGANIZATIONAL  OBJECTIVES  &  PRIORITIES 
CONTENT 

1  Modify  previously  established 

objectives  El  E2 

2  Identify  and  analyze  alternative 

courses  of  action  E1,e2 

3  Determine  priorities  E1,E2 

4  Do  you  feel  that  the  objectives 
developed  as  a  result  of  AMA 
training  reflect  *e  most  seri- 
ous and  pressing  needs  of  state 

education?  E1,E2 


QUESTIONNAIRE 

5  The  goals  of  this  orgemization 

are  articulated*  E1,E2 

6  Our  goals  are  realistic  and 

attainable  with  our  best  efforts*  E1,E2 

7  The  top  priority  objectives  of 

state  education  are  clear  to  mo«  E1,E2 

8  I    feel  that  the  objectives 
developed  during  AMA  training 
reflect  the  most  serious^ and 

pressing  needs  of  state  education.  E1|E2 
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SUiWJiy  OP  ATTITUDINAL  FINDINGS  (cont'd) 


ERIC 


Item  Type  of  Data 

9    As  I  see  It,  the  operation 
priorities  of  the  objectivos 
developed  during  AMA  train- 
ing are  clear* 


Positive     No  Negative 
Effect    Effect  Effect 


4 
5 


E1,E2 


MOBILIZATION  OF  ORGANIZATIONAL  PLANNING 
CONTENT 

Define  standards  of  perfojnnance 
for  key  administrators 

Specify  task  completion  dates 
and  action  assignments 

Assign  responsibilities  to 
subordinate  units 


E2 


E1,E2 
E1,E2 
El 


Need  for  Performance  St£mdards 

Performance  Standards—Extent 
of  Use 


6  Existence  of  Performance  Reviews 

QUESTIONNAIRE 

7  My  organization's  policy  state- 
ments are  clear* 


INSUFFICIENT  DATA 
(T4  ONLY)  ON  WHICH 
TO  BASE  A  RESPONSE 

E1,E2 


8  My  organization's  performance 
standards  are  understood* 

9  Good  ways  are  used  to  let  me 
know  how  I  can  improve  my 
perfoirmance* 

10    I  understand  what  results  must 
be  produced  to  achieve  the 
stated  objectives  of  this 
organization. 

ROLE  OF  THE  PLANNING  UNIT 
CONTENT 

1  Awareness  of  need  to  ovalu-- 
ate  our  programs 

2  Available  to  answer  planning 
questions 

3  Writing  guidelines  for  plan 
development 

4  Reviewing  and  refining  plans 

5  Provides  leadership  in  the 
implementation  of  planning 

6  Provides  in-service  training 
in  planning 


E1,E2 


E1,E2 


E2 


El 


E1,E2 

E1,E2 

El 
E1,E2 

E1,E2 

E1,E2 


E2 
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SUMMARY  OF  ATTITUDINAL  FIUDINGS  (cont'd) 


Item        Type  of  Data   

QOESTIONHAIRE 

7    The  planning  unit  has  been 
helpful  to  me. 

DEVELOFMENT  OF  EVALUATION  TECHNIQUES 
CONTENT 

1    Designed  a  methodology  by  which 
future  perfonncince  may  be  evalu- 
ated in  relation  to  the  perfor- 
mance specified  in  the  plan. 

Informal  Feedback 

Performance  Reviews 

Questionnaires 

Task  Completion  Inventories 

Unobtrusive  Measures 


Positive       No  Negative 
Effect     Effect  Effect 


2 
3 
4 
5 
6 


8 


E1,E2 


E1,E2 
INSUFFICIENT 
DATA  (T4  ONLY) 
ON  WHICH  TO 
BASE  A 
RESPONSE 


QUESTIONNAIRE 

I  have  good  ways  for  knowing 
how  good  our  results  are. 


E2 


My  organization  has  reliable 
ways  for  knowing  how  well  it 
is  attaining  its  objectives. 

I  think  that  the  objectives 
developed  during  AMA  training 
are  clesirly  stated  with 
respect  to  results  expected. 

TOP  MANAGEfiENT  SUPPORT  FOR  PLANNING 

CONTENT 

Adequate  Resources  (money 
and  information) 

Control  System  expressed 
through  decisionmaking  process 

QUESTIONNAIRE 

My  manager  makes  it  clear  he 

is  committed  to  the  success 

of  our  projects.  El 

My  manager  has  expressed  the 
belief  that  the  AMA's  training 
program  has  been  helpful. 

My  manager  understands  planning 
theory  and  is  able  to  put  it 
into  practice. 


El 


E1,E2 


El,E2 


E1,E2 
E1,E2 


E2 


E1/E2 
El,E2 
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SUMMTJIY  OF  ATTITUDINAL  FI^?DINGS  (cont'd) 


Item 
6 


1 
2 
3 


Type  of  Data 

I  believe  my  organization  gives 
me  adequate  training  to  do  my 
work  effectively. 

I  feel  good  cibout  my  manager's 
ability  to  plan. 

My  manager  provides  me  with 
adequate  support  to  perfonr. 
my  job. 


Positive 
Effect 


No 
Effect 


E1,E2 


E1,E2 


Negative 
Effect 


El 


E2 


THE  CREDIBILITY  OF  THE  PLANNING  PROCESS 
CCWTENT 

Establish  credibility  of  planning 

Role  of  Planning:    how  integral  E1,E2 

Role  of  Planning:    how  much 
is  needed 

Role  of  Planning: 


El 


E1,E2 


E2 


emergence 


E1,E2 


QUESTIONNAIRE 


5  As  I  see  it,  planning  is  an 
integral  part  of  running  the 
state's  schools. 

6  As  I  see  it,  persons  in  this 
organization  put  a  lot  of 
effort  into  planning. 

7  My  capability  to  plan  effec- 
tively will  positively  affect 
my  future  career  in  this 
organization. 

8  The  activities  relating  to 
planning  are  having  an  effect 
on  the  policy  of  this  organi- 
zation. 


E1,E2 
E1,E2 


E1,E2 


El 


E2 
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Action  and  Attitudinal  Findings;  SED's 

An  analysis  o£  the  actual  results  in  terzns  o£  written 
orgemizational  doctunents  that  were  produced  in  efforts  to 
meet  the  first  thirteen  American  Management  Association's 
training  criteria  indicates  that,  with  a  few  exceptions, 
the  State  Education  Departments  have  made  moderate  progress 
toward  all  criteria.    The  rese  '  n  believed  that  most 

of  the  action  taken  on  most  o<         c.xceria  illustrated 
that  the  organizations  in  questxon  had  done  more  than  just 
address  the  criterion  (minimum  progress)  but  had  not  actually 
completed  the  criterion  (meucimum  progress) •    The  initiation 
of  a  satisfactory  planning  element  was  present,  however. 

Reviewing  all  criteria,  the  research  team  deteinnined 
five  instances  warranting  a  minimum  rating  and  seven  items 
where  the  criteria  were  met. 

A  summary  of  the  attitudinal  findings  related  to 
these  causal  variables  in  the  SED's  reveals,  again  with  a 
few  exceptions,  a  pattern  of  no  effects  on  attitudes  as  a 
result  of  the  AI4A  training  effort. 

Comparing  the  action  and  attitudinal  findings  related 
to  AMA  criteria/goals  #1  and  #2,  we  find  that,  while  both 
organizations  spent  considerable  time  developing  these 
statements,  no  apparent  effect  on  attitudes . toward  them 
was  observed  within  the  organizations.    In  fact,  there  were 
two  negative  effects  recorded  in  Stat€.  E2.    There,  managers 
felt  less  cleat  about  their  mission  two  years  after  train- 
ing than  before.    They  also  thought  less  of  the  contribution 
they  were  making  to  education  after  training  than  before. 

Moving  on  to  7\KA  criteria  #3  through  #7  (the  develop- 
ment of  specific  objectives  and  priorities),  we  again  find 
a  pattern  of  moderate  progress  on  the  action  side  of  the 
ledger  and  no  effects  of  training  on  attitudes,  with  three 
exceptions.    Only  State  E2  received  a  maximum  rating  con- 
cerning the  degree  to  which  priorities  had  been  determined. 
In  this  state  the  priorities  wore  widely  discussed  and 
published. 

Three  negative  effects  appear  on  the  attitudinal 
data.    Two  of  these  occur  in  State  E2,  which  showed  negative 
training  effects  concerning  the  specific  objectives  and  the 
determination  of  priorities.    While  State  E2  has  determined 
its  priorities,  this  shows  that  some  problem  with  them  has 
arisen  within  the  organization.    The  negative  effect  in 
State  El  appears  to  be  the  result  of  a  lack  of  clarity 
within  the  agency  concerning  its  priorities. 
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7\J1A  criteria  #8  through  #10  are  concerned  with  what 
we  have  tcinned  the  Mobilization  of  Organizational  Planning. 
These  criteria  arc  specifically  related  to  developing 
performance  standards  for  personnel f  assigning  responsibili- 
ties and  setting  task  completion  dates, 

A  clear  difference  between  the  states  emerges  concern- 
ing actions  on  these  items.    State  El  achieved  minimal 
progress  on  all  three  of  these  items  while  State  E2  made 
moderate  progress.    This  reflects,  as  we  have  discussed  in 
Chapter  Four,  a  clear  decision  by  the  agencies.    State  El 
decided  to  place  maximum  emphasis  on  the  development  of 
written  guidelines  for  planning,  while  State  E2  stressed 
development  of  job  descriptions. 

The  attitudinal  data  on  these  items  indicates  that, 
generally,  training  had  no  effect.    Two  positive  effects 
were  observed:    manager sTn  both  states  were  less  clear 
about  their  performance  standards  after  training  than  before 
but  because  their  awareness  did  not  drop  as  fast  as  the 
awareness  of  managers  in  the  Control  State,  we  have  treated 
this  as  a  positive  effect.    A  negative  training  effect 
appears  in  State  El,  where  respondents  grew  more  uncertain 
about  expected  results  as  time  passed.    Again,  this  may  be 
related  to  the  lack  of  clear  performeince  standards  in  that 
state. 

States  El  and  E2  made  considerable  r^rogress  with 
supplementary  planning  efforts  (Item  #11) ,  but  only  moderate 
progress  in  assigning  responsibilities  to  subordinate 
units  (Item  #12).    Both  states  have  been  involved  in  a 
variety  of  activities  designed  to  supplement  their  planning 
efforts. 

The  attitudinal  data  indicate  no  change  in  attitudes 
toward  the  activities  of  the  planning  unit,  however.  The 
one  exception  to  this  appears  in  State  E2,  where  a  negative 
training  effect  appears  related  to  writing  guidelines  for 
plan  development.    Again,  this  reflects  that  agency's 
decision  to  develop  job  descriptions  before  forming  planning 
guidelines. 

The  thirteenth  goal — the  development  of  an  evaluation 
system — indicates  a  minimal  level  of  progress  in  both  states 
and  reflects  a  common  problem  in  education,  one  which  AMA 
training  did  little  to  correct.    Educational  agencies  (like 
most  organizations) ,  simply  do  not  have  personnel  who  are 
expert  in  evaluation;  as  a  result,  this  aspect  of  the  plan- 
ning process  is  generally  neglected. 

The  lack  of  movement  toward  evaluation  ic  reflected 
in  the  attitudinal  data.    Negative  effects  were  recorded  in 
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both  El  and  E2  concerning  the  establishment  of  an  evaluation 
system*    State  E2  did  experience  a  positive  effect  related 
to  general  knowledge  of  results ,  although  this  gain  is 
only  relative.    Personnel  in  E2  actually  experienced  a 
decline  in  their  knowledge  of  expected  results^  but  they 
did  not  decline  as  fast  as  the  Control  State  and  therefore 
experienced  a  positive  training  effect. 

Attitudinal^data  related  to  top  management  support 
for  planning  and  the  degree  to  which  planning  is  seen  as  a 
credible  process  in'^icatet^  a  number  of  positive  as  well  as 
negative  effects. 

The  eight  items  concerned  with  top  management  support 
for  planning  reveal  a  typical  pattern.    Almost  no  effects 
of  training  appear^  with  a  few  exceptions.    In  two  instances, 
personnel  in  State  El  viewed  their  managers  as  more  suppor- 
tive and  committed  to  subordinates'  work  than  was  true 
earlier.    Ac  the  sam^  time,  personnel  in  State  E2  experienced 
negative  training  effects  on  the  same  items.    This  data 
must  be  moderated,  however,  because  we  lack  pre- training 
data  on  these  items. 

The  credibility  of  organizational  planning  was 
affected  by  the  AMA  training  program.    Six  positive  effects 
and  six  negative  effects  were  revealed  by  this  data. 

The  AMA  program  did  enhance  the  credibility  of  plan- 
ning in  the  Experimental  States.    Plajining  came  to  be  seen 
as  more  integral  and  as  increasingly  important  to  the  state 
agencies.    However,  no  effect  was  shown  on  attitudes  about 
how  much  planning  was  needed.    Top  administrators  in  E2  also 
seemed  to  feel  that  planning  became  less  credible  with  time. 

The  questionnaire  items  that  appear  here  reveal  more 
negative  than  positive  effects.    However,  the  lack  of  pre- 
training  data  for  these  items  prompts  us  to  be  careful  in 
assigning  effects.    Over  time,  personnel  in  both  states  saw 
planning  as  being  more  important  to  their  caroercj.    On  the 
other  hand,  they  saw  people  within  the  organizations  putting 
less  effort  into  planning  with  the  passage  of  time.  They 
also  seemed  to  grow  more  aware  that  planning  was  little 
used  in  the  school  system  as  a  v/holo. 
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SUMIlTiRY  OF  ACTION  FINDINGS 
LEA  LEVEL 
AMA  CRITERIA  1-13 
FALL,  1970  to  SPRING,  1972 


AMOUNT  OF  PPOGRESS 


ERIC 


No.  Criteria 

1.  Agreed  upon  a  definition  of 
the  institution's  mission 

2.  Established  continuing . ob- 
jectives and  planning  pro- 
cedures for  long-range 
achievement  of  the  insti- 
tution's mission 

3.  Identified  resources  and 
constraints 

4.  Differentiated  between 
where  the  institution  is 
going  and  where  it  wants 
to  go 

5.  Modified  previously  estab- 
lished objectives 

6.  Identified  and  analyzed 
alternative  courses  of 
action 

7.  Determined  priorities 

8.  Made  strategic  action 
a^ssignments 

9.  Defined  standards  of  per- 
formance for  key  adminis- 
trators 

10.  Specified  task  completion 
dates 

11.  Designed  supplementary 
planning  efforts 

12.  Assigned  responsibilities 
to  subordinate  units 

13.  Designed  a  methodology  by 
which  future  perfonnance 
may  be  evaluated  in  rela- 
tion to  the  performances 
specified  in  the  plan 


Minimum     Moderate  Maximum 

EL1,EL3, 
EL3,EL4 


EL3 


EL3 

EL1,EL2, 
EL3,EL4 

EL3 

EL1,£L2, 
EL3,EL4 


EL3 


EL1,EL2, 
EL3,EL4 


ELl,rL2, 
FL3,EL4 


EL1,EL2, 

EL4 

EL1,EL2, 
EL3,EL4 

EL1,EL4 


EL2,BL4 


EL1,EL2, 
EL3,EL4 


EL2 


ELI 


EL1,EL3  EL2,EL4 

EL1,EL2, 
EL4 

EL1,EL2, 
EL3,EL4 
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SUMMARY  OF  ATTITUDIN^Jj  FINDINGS 

DATA  SUM^i^^RY 

CAUSfJi  VARIABLES  -  LOCAL 
EDUCATION  AGENCIES 

Fall,  1970  -  Spring/  1972 


IMPACT  OF  TRAINING 


Positive         No  Negative 
Item  Type  of  Data  Effect       Effect  Effect 

DEVELOPMENT  OF  ORGANIZATIONAL  MISSION  &  OBJECTIVES 
QUESTIONNAIRE 

1  The  goals  of  this  organization  EL2,EL3, 
are  articulated.  ELI  EL4 

2  Our  goals  are  realistic  and 
attainable  with  our  b^st 

efforts •  ELI ,EL2      EL3 , EL4 

3  The  top  priority  objectives 

for  state  education  are  EL1,EL2, 
clear  to  me,  EL3/EL4 

4  I  feel  that  the  objectives 
developed  during  TdiA  training 
reflect  the  most  serious  and 

pressing  needs  of  state  EL1/EL2# 

education.  EL3 , EL4 

5  As  I  see  it,  the  organiza- 
tional priorities  of  the 

objectives  developed  during  EL1,EL2/ 
JiMA  training  are  clear.  EL3,EL4 

6  The  kinds  of  things  I  am  do- 
ing will  make  a  long  term  EL1/EL2, 
contribution  to  education.  EL3,EL4 

7  As  I  see  it,  my  organization 

is  moving  in  the  right  EL1,EL2, 
direction •  EL3 , EL4 

MOBILIZATION  OF  ORGANIZATIONAL  PLANNING 
QUESTIONNAIRE 

1  My  organization's  policy  EL1,EL2, 
statements  are  clear,  EL3,EL4 

2  My  organization's  performance 
standards  are  clear. 

3  Good  ways  are  used  to  let  me 
know  hov  1  can  improve  my 

performance •  ELI , EL2      EL3 , EL4 
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SUtmRY  OF  ATTITUDINAL  FINDINGS  (cont'd) 

Positive  No  Negative 
Item                 Type  of  Data               Effect       Effect  Effect 

4  I  understand  what  results  must 
be  produced  to  achieve  the 
stated  objectives  of  this  or- 
ganization. EL1,EL2 

5  The  planning  unit  has  been  EL1,EL2, 
helpful  to  me.  EL3,EL4 

6  I  have  good  ways  for  knowing  EL1,EL2, 
how  good  our  results  are.  EL3,EL4 

7  My  organization  has  reliable 

ways  for  knowing  how  well  it  FLlrEL2r 

is  achieving  its  objectives.  EL3rEL4 

8  I  think  that  the  objectives 
developed  during  2\MA  training 

are  clearly  stated  with  re-  ELl^ELS^ 

gard  to  results  expected.  EL3rEL4 

TOP  MANAGEMENT  SUPPORT  FOR  PLANNING 
QUESTIONNAIRE 

1  My  manager  makes  it  clear  he  is 

committed  to  the  success  of  our  EL1,EL2, 

projects.  EL3rEL4 

2  My  manager  has  expressed  the 
belief  that  the  AMA's  training 

program  has  been  helpful.  EL3,E14  EL1,EL2 

3  My  manager  understands  plan- 
ning theory  and  is  able  to 

put  it  into  practice.  ELl,Eli2  EL3^EL4 

4  I  believe  my  organization 

gives  me  adequate  training  to  EL1,EL2, 

do  my  work  effectively.  EL3rEL4 

5  I  feel  good  about  my  manager's  ELlrEL2r 
ability  to  plan.  EL3,EL4 

6  My  manager  provides  me  with 

adequate  support  to  perform  ELlrEL2r 

my  job.  EL3rEL4 

CREDIBILITY  OF  THE  PLANNING  PROCESS 
QUESTIONNAIRE 

1  As  I  see  it,  planning  is  an  in- 
tegral part  of  running  the  ELlrEL2, 
state *s  schools*  EL3,EL4 

2  As  I  see  it,  persons  in  this 

organization  put  a  lot  of  effort  EL1,EL2, 

into  planning.  EL3  EL4 
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SUMMARY  OF  ATTITUDINAL  FINDINGS  (cont'd) 


Positive 


No 
Effect 


Negative 
Effect 


Item  Type  of  Data  Effect 


3    My  capability  to  plan  effec- 
tively will  positively  affect 
my  future  career  in  this  or- 
ganization. 


EL1,EL2, 
EL3,EL4 


4    The  acrivities  relating  to 
plemning  are  having  an  effect 
on  the  policy  of  this  organi- 
zation. 


EL1,EL2, 
EL3,EL4 


Action  and  Attitude  Findings/LEA' s 

Where  the  Local  Educational  Agencies  (LEA's)  are  con- 
cerned, Chapter  Four  was  to  assess  the  impact  of  7\MA  training 
on  the  causal  variables  associated  with  organizational  change. 
The  thirteen  AMA  training  criteria  were  reviewed  and  applied 
to  the  agencies  to  determine  the  extent  to  which  the  goals 
were  realized.    We  etnalyzed  the  attitudes  of  participants 
toward  the  development  of  orgamizational  mission,  mobiliza- 
tion of  organizational  planning,  top  management  support  for 
planning,  and  the  credibility  of  the  planning  process. 

The  inventory  taken  in  connection  with  the  thirteen 
AMA  criteria  revealed  that  all  had  been  addressed  by  the 
management  teams  from  the  LEA's,  who  had  written  something 
on  each  of  the  criteria.    The  research  team  felt  that  most 
of  the  actions  taken  on  most  of  the  criteria  could  be  termed 
"moderate"  progress;  that  is,  the  organization  had  done 
more  than  merely  address  the  criterion — it  had  initiated  a 
satisfactory  element  of  successful  planning. 

If  the  agencies  had  been  able  to  go  beyond  a  beginning 
to  approach  completed  action  on  the  criterion,  their  progress 
would  have  been  "maximum."    Two  criteria  fell  into  the 
latter  category:    a  definition  of  the  institution's  mission 
and  its  establishment  of  continuing  objectives.    The  LEA's 
presented  both,  fully  developed,  in  their  planning  documents* 
On  the  opposite  end  of  the  spectrum  were  "minimum"  progress 
on  various  criteria,  those  on  which  virtually  nothing  had 
been  done  beyond  superficial  examination  of  possible  issues 
or  techniques.    Efforts  to  estaiblish  priorities,  performance 
standards,  and  evaluation  methods  were  indeed  minimal  in  all 
the  planning  documents  we  examined. 

The  attitudes  section  of  the  chapter  was  strikingly 
unable  to  attribute  more  than  a  few  training  effects  to  the 
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AMA  program*    The  attitudinal  smnmary  charts  on  the  impact 
of  training  repeatedly  a8signe"3rLEA'8  to  the  middle^  or 
"No  Effects"  column*    On  the  basis  of  the  reliable  instru- 
ments  used  to  measure  effects,  we  conclude  that  the  AMA 
team  plemning  process  did  not  create  a  favorable  attitudinal 
environment  for  planning  activities*    Those  LEA's  that  could 
be  considered  more  favoreJale  before  training  were  similarly 
favorable  a  year  after  training.    And  those  that,  had  been 
more  hostile  to  the  changes  implicit  in  management  by 
objectives  did  not  grow  more  friendly  to  such  management. 


Overall  Conclusion  on  the  Impact  of  AI^A  Training 
on  SED/LEA  Causal  Variables 

It  has  been  argued  throughout  this  report  that 
actions  and  attitudes  are  be  to.  critical  factors  in  evaluat- 
ing  any  training  program.    Actions  directed  toward  implement- 
ing the  planning  process  can  degenerate  into  meaningless 
paper-production  without  the  support  of  people  in  the 
orgemization.    Even  receptive  attitudes  toward  planning  will 
not  long  endure  if  they  are  not  reinforced  by  practical 
steps  tov/ard  its  development  and  use.    It  follows,  therefore, 
that  to  determine  the  value  of  the  AMA  to  these  educational 
agencies  regarding  causal  variables,  one  must  examine  what 
participants  did  as  well  as  what  they  thought  about  what 
had  been  attempted. 

MIA  training  definitely  inspired  the  SED*s  and  LEA's 
to  devote  more  time  and  energy  to  planning.    It  also  pro- 
duced a  wide  range  of  plannitig  materials.    But  the  aggregate 
attitudes  of  those  who  were  expected  to  accept  and  work 
with  the  planning  process  were  tinaffected  by  training.  If 
attitudes  associated  with  the  causal  variables  were  already 
receptive  to  planning  before  training  began,  then  the  AMA 
did  not  significantly  alter  that  vxew  wxthin  the  organiza- 
tion.   On  the  other  hand,  if  persons  felt  confused  about 
their  goals,  alienated  from  their  managers,  or  uncertain 
of  thelbenefit  of  planning,  training  also  did  little  for 
them. 
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CHAPTER  FIVE 


INTERVENING  VARIABLES 


Overview 

This  chapter  evaluates  the  impact  of  AMA  training 
on  the  intervening  variables  in  the  experimental  education 
agencies,  variables  which    reflect  the  internal  state  and 
health  of  the  organization,  e.g.,  the  loyalties,  attitudes, 
motivations,  performance  goals,  and  perceptions  of  all 
members  and  their  collective  capacity  for  effective  inter- 
action, communication,  and  decision  making* These 
variables  are  the  middle  link  in  the  causal  -  intervening  ^ 
end  result  variable  chain. 

Theoretically,  implementing  the  AMA  training  goals 
(causal  variables)  should  affect  the  decisionmaking  pro* 
cess,  leadership  climate,  and  management  team  relations 
in  the  educational  agencies  (intervening  variables) . 
Those  internal  changes  should  correlate  pcsitively  with 
increases  in  the  quantity  and  quality  of  the  organization's 
output,  particularly  in  plamning  (end-result  variables). 

Lack  of  successful  impact  upon  intervening  variables 
can  impede  or  even  prevent  a  training  program  from  improv- 
ing overall  planning  and  output.    Therefore,  such  variables 
should  be  important  to  the  AMA  training  program,  and  a 
proper  concern  of  this  evaluation. 

Our  examination  is  divided  into  four  sections.  The 
first  section  looks  at  the  AI1A  approach  to  the  intervening 
variables.    In  the  second  section,  we  study  the  7\MA's 
impact  on  these  variables  through  the  expressed  attitudes 
of  ESED  participants.    The  consequences  for  the  ELEA's  is 
shown  through  another  attitudinal  data  display  in  the  third 
section.    Based  on  the  preceding  three  sections,  the  fourtn 
offers  the  overall  conclusions  of  the  research  team  on 
this  aspect  of  AMA  training. 


^^Rensis  Likert,  The  Human  Organization  (New  York: 
McGraw*-Hill  Book  Company,  1967),  p.  29. 


296 


Section  It    The  AMA  Approach 


Wo  have  already  discussed  at  some  length  the  AMA's 
general  approach  to  management  training.    At  this  point 
we  need  only  extract  from  previous  sections  data  relevcint 
to  the  intervening  variables,  and  review  them  here.  The 
presentation  of  SED/LEA  participsmt  attitudes  and  our  own 
analysis  follows. 

Assuming  a  rational  process  of  decisionmaking,  the 
2\MA  seeks  to  win  the  commitment  of  the  chief  executive  of 
the  educational  agency  to  planning,  and  to  involve  his 
immediate  subordinates  in  developing  a  long-range  action 
plan*  ^    AtlA  argues  that  its  program  is  not  just  an  academic 
exercise  but  a  real-life  situation  in  which  hard  decisions 
about  the  organization's  future  are  made  by  the  management 
team.-'    Planning  is  learned  by  doing. 

AMA  has  a  unique  solution.    It  is  called 
the  Team  Process  in  Corporate  Plaiuiing. 


This  process  was  developed  at  the  Center  t 

for  Planning  and  Development  of  the  \ 

American  Management  Association  at  Hamil**  1 

ton.  New  York.    In  it  the  chief  executive  | 

officer  (with  his  final  authority)  and  4 

the  members  of  the  top  management  team  5 

(with  decision  making  responsibilities  J 

in  over-all  company  operations)  proceed,  5 

step  by  step,  through  a  guided  per for-  | 

mance  of  the  planning  procedure.    The  | 

.  team  moves  quickly— it  learns  by  doing —  j 

and  in  two  spaced  out  weeks  it  has  pro-  ? 
duced  its  own  action-oriented  company 
plan."^ 


2 

Lawrence  A.  Appley,  •'Manager  Training  in  Proper 
Perspective,"  in  AMA,  "AtlA:    A  Brief  Description"  (pamphlet). 

Raymond  E.  Klawuhn  and  Alexander  J.  Basso,  "Pinal 
Report:    Adapting  and  Testing  Business  Management  Develop- 
nent  Programs  for  Educational  Administrators"  (January, 
1972),  p.  7.    It  is  interesting  to  note,  however,  that, 
according  to  the  AMA,  participants  still  thought  of  the 
training  as  training  and  not  as  an  organizational  planning 
process.    See  Klawuhn  and  Basso,  0£.  cit. ,  p.  35.  i 

A  i 

Center  for  Planning  and  Development,  American  Man-  i 

agement  Association,  "Developing  and  Iittplementing  Strategic  ] 

Q                 Long-Range  Planning"  (pamphlet) .  I 

ERJC  1 
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implicit  in  the  AMA's  approach  is  the  assumption 
that  this  team  planning  process  will  insure  commitment  to 
the  plans  produced,  since  the  people  who  created  the  plans 
will  implement  them.    The  element  of  team  is  emphasized  as 
a  vehicle  to  participative  decisionmaking  and  an  improved 
organizational' climate.    Given  these  expected  positive 
internal  changes,  plans  and  programs  toward  the  objectives 
the  team  sets  are  assumed  to  be  favorably  affected. 

The  AMA's  approach  relies  completely  upon  a  cognitive 
change  strategy.    They  deal  only  with  technical  knowledge 
of  planning  and  other  aspects  of  developing  and  executing 
an  organizational  plan.    The  AMA.  assumes  that  by  giving 
managers  a  cognitive  awcureness  of  the  benefits  of  plcinning, 
instructing  them  in  how  to  plan,  and  taking  them  through 
the  team  planning  process,  they  will  automatically  gain 
their  allegiance  as  well  as  support  throughout  the  organi- 
zation. 

Theso  assumpcions  overlook  affective  aspects  of  the 
planning  process,  such  as:  . 


Receiving  and  giving  nonevaluative  feedback 
Owning  and  permitting  others  to  have  their 
own  ideas,  feelings  and  values 

Openness  to  new  ideas,  feelings,  and  values 
Experimentation,  risk-taking,  and  new  ideas 
and  values. 5 

Trust,  honesty,  supportive  personal  relationships  and  other 
ingredients  of  a  proper  mix  of  intervening  variables  are 
not  necessarily  acquired  through  a  rational  process  of  team 
planning.    A  viable  mix  of  rationalism  and  affective  con- 
siderations demands,  among  other  things,  an  appreciation 
of  and  expertise  in  organizational  behavior  and  dyneunics. 
AMA  training  deals  with  these  elements  indirectly,  if  at 
all.    Learning  that  planning  is  useful  and  possible  may 
affect  a  client's  attitude  tov/ard    the  concept  (albeit 
temporarily),  but  it  will  not  benefit  him  or  the  agency 
in  the  long-run  if  his  boss  is  personally  antagonistic,  his 
colleagues  ruthlessly  competitive,  top  management  perceived 
as  incompetent,  or  if  fear  rather  than  trust  dominates  the 
organizational  environment.    Even  team  training  accomplishes 
little  if  conflicts  are  based  on  subjective  tensions  that 
are  unrelated  to  the  training's  objective  content. 


Chris  Argyris,  Intervention  Theory  &  Methods  A 
Behavioral  Science  View  (Reading,  Mass.:  Addison-Wesiey 
Publishing  Company,  19^0) ,  p.  66. 
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I  perceive  a  real  facade  and  resistance 
to  laying  all  the  cards  on  the  table 
when  this  group  meets.  .  .  .  i  don't 
think  this  kind  of  planning,  a  team 
planning,  is  going  to  work  until  you 
can  get  people  to  really  think  like 
they're  on  a  team.    And  they  sure  didn't 
make  much  progress  on  that  score  up  at 
Hamilton.^ 

Very  little  of  the  AMA  approach  addresses  these  complex 
aspects  of  organizational  life.    This  gap  gravely  decreases 
the  program's  impact  on  the  intervening  variables. 


Section  2t    SEP  Attitudes 


The  data  presented  in  this  section  measure  the 
current  state  of  the  intervening  variables  in  the  State 
Education  Departments  (SED)  that  participated  in  the  AMA 
program.    Our  specific  concerns  in  this  area  have  been 
organized  under  three  categories: 

A.  Leadership  Climate 

B.  Decisionmaking 

C.  Management  Team  Relations 

Leadership  climate  refers  to  the  environment  fostered 
within  the  organization  by  those  in  authority.    How  deci- 
sions are  made  in  the  LEA's  is  a  second  importanFTntervening 
variable;    Finally,  the  AMA  team  planning  process  depends 
upo'i  a  management  team.    The  extent  to  vrtiich  such  a  team 
exists  ana  maintains  good  relations  indicates  how  much 
viable  planning  will  be  done. 

T:iis  section  includes  content  analysis  and  question- 
naire data.    Many  of  the  items  in  both  instnunents  cover 
four  points  in  time  equally  spaced  between  Pall  1970  and 
Spring  1972.    Other  items  are  tested  at  two  points  in  time. 
Some  of  the  comparisons  of  questionnaire  data  are  made  be- 
tween two  experimental  groups;  most  compare  the  experimental 
group  with  one— if  not  two— control  group (s).    The  research 
team  has  greatest  confidence  in  items  that  span  four  points 
in  time  and  involve  a  control  group  comparison.    In  those 
cases  where  optimal  comparisons  were  impossible,  we  were 
especially  cautious  in  our  interpretations.    In  any  event, 
the  effects  which  we  did,  or  did  not,  attribute  to  AMA 
training  are  the  outcomes  of  careful— and,  we  feel, 
accurate — analysis . 


ERIC 


Consultant/Specialist,  ESED  #2. 
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Throughout  this  section,  content  analysis  data 
will  be  presented  first,  in  each  category •    These  reflect 
the  reactions  of  the  top  management  of  the  two  Experimental 
SED's  (El  and  E2)  and  the  first  Control  State  (CI).  In 
displaying  the  content  analysis  data,  four  kinds  of  infor- 
mation are  provided: 

1.  Interview  Question;    Prom  what  question  is 
the  data  drawn? 

2.  Range  of  Scale  Possibilities:    fThile  the  num- 
ber of  points  on  the  scales  is  constant  (7),  the  descriptive 
words  attached  to  points  on  the  scale  vary  with  the  question. 

3.  Point  of  Time;    For  each  item  the  points  in 
time  at  which  the  question  was  posed  are  indicated.     (Tl  = 
Pall  1970;  T2  =  Spring  1971;  T3  *  Pall  1971;  T4  =  Spring 
1972.)     In  some  cases  this  includes  all  four  points  in 
time;  in  others  the  data  v;ere  gathered  durinfcf*Y2  only. 

4.  States;    The  states  to  which  the  category  was 
applicable  are  identified. 

Relev2mt  c[uestionnaire  items  have  been  sampled  from 
multiple  levels  o^  the  agencies  and  follow  the  content 
data.    These  include  comparisons  with  two  Control  States 
(CI  and  C2)* 

In  the  case  of  the  data  from  the  questionnaire  we 
need  to  indicate  only  the  points  in  time  to  which  the 
question  is  relevant.    The  range  of  scalar  possibilities 
was  in  all  cases  the  some,  from  (1)  not  at  all  to  (7)  very 
often.    As  indicated  above,  questionnaire  items  were 
askod  in  Control  state  #2  in  points  of  time  T3  and  T4 
only,  because  this  state  entered  the  research  design  in 
Year  2. 


h%    Leadership  Climate 

The  following  items  all  come  from  the  questionnaire 
and  were  asked  in  all  four  periods. 

1.  Based  on  information  I  have  received  from 

my  boss,  I  know  if  I  am  measuring  un  in  my  job. 

2.  My  manager  encourages  and  supports  innovation. 

3.  Higher  management's  reactions  to  the  problems 
that  reach  them  are  fair; 

4.  My  manager  knows  and  understands  the  problems 
I  face. 
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!ly  manager  recogn'  zes  when  a  problem  is 
developing  and  does  something  constructive 
about  it. 

My  manager  shows  confidence  and  trust  in  me. 
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Item  11:    Based  on  infonnation  I' have  rocoived  from 
my  bosSy  I  knov/  if  I  am  measuring  up  in  my  joST 

Fall  1970  to  Spring  1971 

In  all  the  states  the  mean  scores  on  this  item 
dropped,  between  Tl  and  T2.    No  training  effect  v/as  shoi/n 
in  State  Bl  although  it  was  significantly  different  from 
Control  1  in  the  amount  of  feedback  given  by' the  manager. 

State  E2  showed  a  positive  training  effect*  It 
differed  from  the  Control  State;  though  both  states  were 
declining  on  this  item  State  E2  experienced  less  decline 
and  a  greater  stabilization  of  opinion  than  did  Control  1. 

Spring  1971  tp  Fall  1971, 

No  significant  differences  appeared  here  between 
State  El  and  Control  1*    State  E2  did  not  change  over 
time  in  comparison  with  Control  1  but  continued  to  differ 
significemtly  from  CI  in  terms  of  the  level  of  response 
to  this  item* 

Fall  1971  to  Spring  1972 

There  were  no  significant  training  effects  during 
this  period.    No  significant  differences  appeared  between 
State  El  and  Control  1,  or  betv^een  State  E2  and  Control  2. 
Significant  differences  between  State  El  and  Control  2 
and  between  State  r2  and  Control  1  were  registered  during 
this  period. 

Fall  1970  to  Spring  1972 

An  analysis  over  time  indicates  significant  positive 
training  effects  in  Fl  and  negative  effects  in  E2. 

State  El  exporienced  a  significantly  less  rapid 
decline  on  this  item  than  the  Control '.State* /.oPerdonncl  in 
El  received  less  feedback  from  their  managers #  but  at  a 
slov/cr  rate  of  decline  than  did  organization  mend^ers  in 
Control  1. 

State  E2  staff  experienced  a  lower  rate  of  feedback 
at  a  rate  of  decline  which  was  greater  than  that  cf  Control 
1.    However,  attitudes  on  this  subject  were  incr<.asingly 
stabilized  in  State  E?,  while  they  v;erc  growing  more 
unstable  in  Control  1. 
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Item  #2:    :iy  manager  oncourages  and  supports 
innovation . 

Fall  1970  to  Spring  1971 

Mo  Gf foots  attributable  to  training  wsre  producod 
in  E2  during  the  rro/post-training  periods.    VGry  signifi- 
cant effects  were  recorded  in  rl.    Both  of  the  states 
were  significantly  different  from  the  Control  State  in 
terms  of  the  amount  of  encouragement  and  sunport  given  to 
innovation;  support  in  th^se  terms  fran  managers  in  El  was  \ 
greater  than  encouragement  given  in  the  Control;  support  | 
in  E2  was  less  than  that  given  in  the  Control.  | 

i 

Spring  1971  to  Fall  1971  \ 

No  training  effects  occurred  in  this  period  of  time.  4 
State  El  and  the  control  continued  to  be  significantly 
different  from  each  other,  while  this  difference  between  | 
E2  and  the  Control  faded.  j 

I 

Fall  1971  to  Spring  1972  I 

i 

There  was  no  significant  change  in  the  support  for  \ 
innovation  in  El  during  ¥2,  although  the  difference  between  : 
El  and  Control  1  continued.  El  ';as  also  significantly  i 
different  from  C2. 

Significantly,  negative  training  effects  did  occur 
in  E2.    In  comparison  with  both  Control  groups,  E2  personnel 
experienced  a  sharper  decline  in  the  degree  of  innovation 
supported  by  their  managers. 

Fall  1970  to  Spring  1972 

A  'P1-T4  cOTinarison  reveals  a  significantly  negative 
effect  of  training' in  E2.    Both  El  and  E2  experienced 
weaker  support  for  innovation  and  more  instability  concern- 
ing those  attitudes  in  the  Spring  of  1972  than  was  true 
in  the  FaJ.l  of  1970,  before  training.    In  El's  case,  the 
decline  was  lafss  than  that  undergone  by  the  Control  State 
over  the  samp  period;  therefore  a  positive  effect  was 
assigned.    E2''s  negative  effect  came  about  as  the  result 
of  its  decline,  much  greater  than  the  Control  State  s. 
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Item  f^3:    Fall  1970  to  Spring  1971 

When  Eighcr  management's  reactions  to  the  problems 
that  reach  them  cire  fair  is  assesserl^  no  training  effect 
was  evinced  in  State  El.    State 'E2  experienced  a  positive 
training  effect^  because  more  stabilization  concerning 
attitudes  toward  higher  management's  reactions  was  seen 
there  than  in  the  Control  State.    Neither  El  nor  E2 
differed  from  the  Control  in  strength  of  belief  that  top 
management's  reactions  were  equitable. 

Item  ^3;    Spring  1971  to  Fall  1971 

It^  cannot  fairly  be  argued  that  training  had  any 
effect^  one  way  or  the  other ^  on  the  opinions  that  El  or 
E2  held  about  the  way  top  management  dealt  with  organiza- 
tional problems.    Both  remained  relatively  stable  in  their 
beliefs,  although  CI  was  less  favorably  disposed  toward 
its  superiors  than  was  El.    '^Tanagers  in  El  and  CI  reacted 
in  the. same' way  toward  issues  which  came  to  their  attention 
during  tixis  ^period. 

T"       Item  #3;     Fall  1971  to  Spring  1972 

El  felt  that  higher  management's  reactions  were 
fairer  in  their  organization  that  CI  did  although  there 
was  no  difference  between  the  Experimental  States  and 
Control  States.    In  neither  case  did  change  occur  between 
Fall  1S71  and  Spring  1972.    However,  CI  and  C2  had  a  higher 
opinion  of  their  management's  handling  of  problems  than  did 
E2,  and  these  changes  over  time  were  at  significant  levels. 
Reaction  became  more  unfair  in  all  three  States  between  T3 
and  T4r  but  E2's  dovmward  trends  were  steeper  than  in 
either  Control  State.    Therefore  a  negative  effect  of 
training  was  concluded. 

Item  »3:    Fall  1970  to  Spring  1972 

ITo  impact  from  the  weeks  snent  learning  planning 
processes  and  developing  action  plans  vras  evident  in  El 
higher  management's  reactions  to  problems  that  reached 
them.    The  differences  between  El  and  CI  on  this  variable 
were  insignificant.    Those  who  responded  in  E2  sensed  a 
grooving  inability  of  their  managers  adequately  to  attack 
educational/administrative  questions;  and  yet  the  growth 
in  CI  was  greater.    If  wo  assume  that  the  presence  of  WiA 
training  in  E2  and  its  corresponding  relative  absence  in 
CI  made  the  difference  in  their  development  of  management 
problem-solving  skills,  then  the  AMA  program  fostered  a 
positive  change  in  E2. 
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Item  »4:    Fall  1970  to  Spring  1971 

Data  gathered  be£ore  and  after  training  on  the 
extent  to  which  My  manager  knows  and  understands  the  prob- 
lems I  face  resulted  jn  different  training  effects.  In 
the  case  of  El,  different  effects  meant  no  effect;  mana- 
gerial knowledge  of  subordinate  problems  did  not  change 
over  time  and  was  the  same  as  found  in  CI.    In  E2, 
"different"  implies  negative  effects.    While  both  E2  and 
the  Control  State  declined  between  the  Fall  of  1970  and 
the  Spring  of  1971,  E2  declined  faster  than  CI  and  thus 
experienced  a  negative  training  effect.  Significant 
differences  between  E2  and  CI  were  also  observed. 

Item  »4:    Spring  1971  to  Fall  1971 

El  and  CI  remained  constant  in  their  essential 
similarity;  managers  in  El  did  not  know  any  more  or  under-* 
stand  any  more  acutely  than  their  counterparts  in  CI. 
Knowledge  and  understanding  did  increase  (although  not 
significantly)  in  F2  when  compared  to  CI,  but  neither 
changed  sufficiently  over  time  to  assume  any  effects  of 
the  AI?A-*induced  planning  process. 

Item  #4:    Fall  1971  to  Spring  1972 

Training  had  a  negative  effect  on  E2  between  Pall 
1971  and  Spring  1972.    Managers  in  that  Experimental  State 
knew  less  and  understood  1       clearly  as  a  result  of  the 
Hamilton  experience.    Both  E2  and  CI  declined  over  this 
period  but  CI  did  not  decline  as  sharply  as  E2  despite  the 
lack  of  AMA  training  in  the  Control  State.    E2  managers 
were  more  skilled  in  dealing  with  problems  than  those  in 
C2,  but  neither  changed  in  T3  to  T4,  so  no  effects  are 
presumed. 

Item  #4:    Fall  1970  to  Spring  1972 

The  ability  of  managers  in  Cl  to  comprehend  and 
handle  developing  organizational  problems  was  not  signifi- 
cantly different  from  the  ability  of  those  in  El.    And,  in 
fact,  managerial  competence  to  handle  problems  fell  in 
both  States  between  Fall  1970  and  Spring  1972.    Since  the 
rate  of  negative  change  was  slower  in  El,  v?e  can  attribute 
positive  effects  to  A!1A  training.    The  opposite  occurred 
in  E2,  since  its  scores  fell  more  than  those  of  Cl,  and 
this  constituted  a  negative  training  effect. 
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Item  ^5!    Fall  1970  to  Spring  1971 

Tho  extent  to  which  Hy  manager  recognizes  when  a 
problem  is  developing  and  does  something  constructive 
about  it  offers  another  variable  rolatea  to  leadership  and 
organizational  climate.    Training  produced  changes  and 
differences  in  both  States.    More  managerial  recognition 
existed  in  E2  before  the  training  than  after  it,  creating 
significant  negative  changes  v;ithin  the  organization,  a 
significant  difference  with  CI,  and  a  negative  effect  of 
AMA  training.    El  also  reported  less  recognition  in  a  pre- 
and  post-training  comparison  with  CI,  but  Cl's  decline  was 
greater #  thus  causing  a  positive  effect  in  El. 

Item  #5;    Spring  1971  to  Fall  1971 

Mo  training  effects  appeared  during  this  period. 
Major  differences  continued  between  the  States;  their  dif* 
ferences  were  the  same  as  pr^^viously  indicated  with  CI 
showing  more  managerial  recognition  of  problems  than  E2, 
but  less  than  El.    No  State  changed  enough  over  time  for 
training  to  have  any  impact,  positive  or  negative. 

Item  »5;    Fall  1971  to  Spring  1972 

El  did  not  significantly  differ  from  CI  in  their 
perception  of  organizational  problem- solving  skills,  but 
El  easily  led  C2  in  this.    On  the  other  hand,  F2  declined 
significantly  between  Fall  1971  and  Spring  1972  and  was  much 
lower  on  this  item  than  either  Control  State.  Therefore, 
training  had  the  effect  of  decreasing  the  degree  to  which 
managers  in  E2  could  recognize  and  deal  with  emergent 
educational  problems. 

Item  #5;    Fall  1070  to  Spring  1972 

The  effects  v/e  observed  in  El  in  an  earlier  period 
evinced  some  permanence.    Our  overall  assessment  concludes 
that,  concerning  this  varicible,  AMA  training  had  a  positive 
effect  on  El.    The  loss  of  management  problem- solving 
skills  in  El  was  not  nearly  so  great  as  in  CI.    The  reverse 
occurred  in  E2,  which  declined  much  more  quickly  and 
steeply  than  the  Control  State.    That  a  variable  could 
perform  this  way  in  a  State  which  had  AMA  training  (as 
opposed  to  a  State  without  training)  indicates  a  negative 
effect  (in  E2)  of  being  trained  by  A^IA  in  the  team  planning 
process • 
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Item  #6;    Fall  1970  to  Spring  1971 

Item  #6^  My  manager  shows  conflc^.ence  and  trust  in 
me.    Thoro  arc  no  significant  differences  between  El  and 
the  Control  State  when  we  compare  what  each  said  about  this 
aspect  of  managerial  relations  before  training  and  after- 
v;ard.    Training  did  have  negative  impact  in  E2;  their 
managers  wore  less  confident  after  training  about  their  sub- 
ordinates* cibility  to  get  the  job  done;  and  the  managers 
wero  losing  this  confidence  faster  than  in  the  Control  State 

Item  #6;    Spring  1971  to  Fall  1971 

No  effects  due  to  AT^.^  training  were  observed  in 
either  State.    No  changes  occurred  over  time  and  no  differ- 
ences appeared  between  El  and  CI,  or  between  E2  and  CI. 

Item  #6;    Fall  1971  to  Spring  1972 

Between  Fall  1971  and  Spring  1972,  El  respondents 
believed  their  managers  had  a  higher  opinion  of  them  than 
their  counterparts  in  CI,  but  neither  had  changed  much. 
An  effect  of  training  did  take  place  in  E2.    In  comparison 
with  CI,  El  dropped  faster  and  farther  in  terms  of  manage- 
ment confidence  and  trust.    This  significant  downward  move- 
ment is  a  negative  training  effect. 

Item  »6;    Fall  1970  to  Spring  1972 

A  comparison  of  pre-  and  post-training  results  on 
this  item  in  both  Experimental  States  reveals  opposite 
training  effects.    El,  E2,  and  CI  all  indicated  less 
expressed  managerial  confidence  and  trust  in  Spring  1972 
than  in  Fall  1970.    l^ereas  the  decline  in  CI  was  faster 
than  in  El,  it  was  slower  than  in  E2.    Without  training, 
v;e  assume  that  all  declines  would  have  been  similar;  like- 
wise we  assiame  that,  with  training,  decline  in  an  Experi- 
mental State  would  be  matched  by  greater  decline  in  the 
Control  State.    That  this  held  true  for  El  is  a  positive 
effect.    That  it  proved  false  for  E2  is  a  negative  impact. 
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DATA  SUMilARY 
Leac'.cjrship  Climate 
Fali,  1970  -  Snrinq,  1972  IMPACT  OF  TRAINIIJG 


Type  of  Data  Positive       No  Negative 

itom  otiESi'iQ|)^^^y:  Effect     Effect  Effect 

1  daeod  on  information  I  have 
received  from  my  bossr  I  know 

if  I  am  measuring  un  in  my  job.       El  e2 

2  fly  manager  encourages  anr*  suo- 

ports  innovation.  ei  e2 

3  Higher  management's  reactions 
to  the  problems  which  reach 

them  are  fair.  E2  El 

4  lly  manager  knows  and  under- 
stands the  problems  I  face.  El  e2 

5  My  manager  recognizes  when  a 
problem  is  developing  and  does 

something  constructive  about  it.     El  e2 

6  My  manager  shows  confidence 

and  trust  in  roe.  El  e2 


Even  a  casual  reading  of  the  data  summary  reveals  the 
AMA  program  significantly  influenced  leadership  climates  in 
the  Experimental  States.    With  one  exception,  training 
effects  were  observed  on  every  questionnaire  item.    And  it  is 
also  clear  that  the  States  were  affected  in  opposite  ways. 

The  leadership  climate  in  El  was  imnroved  largely  by 
the  A'lA  intervention.    ;^s  far  as  our  instruments  were  able 
to  measure,  El  made  the  team  planning  process  less  a  slogan 
and  more  genuinely  an  institutional  reality.    Positive  train- 
ing effects  were  recorded  on  all  but  one  of  the  items  used 
to  assess  the  managerial  environment.    The  educational 
managers  of  El  v;ere  apparently  viewed  by  their  administra- 
tive subordinates  as  improved  information  providers  and 
supporters  of  innovation;  as  understanding  managers,  problem 
solversj  ^d  as  superiors  who  seemed  to  trust  and  confide 
in  subordinates.    Only  in  their  immediate  reaction  to  problems 
did  managers  exhibit  no  movement  in  either  direction. 
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The  situation  was  roverscrl  in  E2.    The  AMA  nrogram 
caused  serious  deleterious  consequences  for  that  organi- 
zation* sTeaSership  climate.    In  the  53 me  items  where  El 
had  experienced  positive  effects^  E2  amassed  negative 
effects.    The  single  exception  was  in  their  responses  to 
problems  that  reached  them;  persons  in  E2  felt  that  mana- 
gerial responses  had  becOTie  fairer  after  training.  But 
in  all  other  areas,  the         program  hurt  more  than  it  helped. 

This  is  not  to  suggest  that  the  positive  effects 
imply  an  improved  climate  in  El;  or  that  the  negative 
assessment  in  E2  should  automatically  be  interpreted  to 
mean  that  this  Experimental  State  was  the  only  one  to  show 
a  decline.    On  the  contrary^  both  Experimental  States  (and 
the  Control  State)  had  lower  mean  scores  after  training 
than  before.    Effects  of  training  were  assigned  on  the 
legitimate  basis  that  one  Experimental  State  (El)  had 
declined  more  slowly  than  the  Control  State,  while  the 
other  State  (E2)  had  fallen  much  faster.'   Methodologically , 
we  assumed  that  if  the  states  had  not  been  trained ,  the 
Experimental  States  would  have  performed  somewhat 
similarly  to  the  Control  State.    To  the  extent  they  did 
not,  the  disparity  was  defined  as  an  effect  of  training. 


B.  Decisionmaking 

The  analysis  of  decisionmaking  is  baser*,  on  three 
content  analysis  items  and  three  questionnaire  items. 

1 .  Involvement  in  Deci'^ionmaking  in  the  State 
Department; 

Interview  Question:    How  are  major  decisions 
made  in  the  State  Department? 

Range  of  Scale  Possibilities:     (1)  no 
participation/no  discussion 
invited i  to  (7)  maximum  partici-- 
pation  throughout  SED. 

Points  of  Time:     Tl,  T2,  T3,  T4. 

States:    El,  E2,  CI. 

2.  Quality  of  Decisionmaking  in  the  State  Departmenc. 

Interview  Question:    Scone  as  #1  above. 

Range  of  Scale  Possibilities:     (1)  never 

effective  to  (7)  highly  effective. 
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Points  of  Time:    Tl,  T2/  T3,  T4. 
States:    El,  E2,  CI. 

3.  Influence  of  Planning  on  c^^ocisionmaking  process. 

Interviev;  Question:    Is  Planning  influencing 
the  decisionmaking  process  within 
the  State  Bopartinent  of  Education? 

Range  of  Scale  Possibilities:     (1)  no  influence 
to  (7)  integral  part  of  decision- 
making process. 

Points  of  Time:    T3,  T4. 

States;     El,  E2,  CI. 

The  questionnaire  items  are: 

4.  The  people  I  work  with  participate  appropri- 
ately in  setting  the  goals  of  our  vforkl 

Points  of  Time:    Tl,  T2,  T3,  T4. 

5.  I  am  appropriately  involved  in  dccisi^^ns 
affectlntj  my  woric! 

Points  of  Time:    Tl,  T2,  T3,  T4. 

6.  I  can  influence  the  goals,  methods,  and 
activities  o^  my  organization. 

Points  of  Time?    T3,  T4. 
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Item  3_:    Influence  of  Planning  on  Decision-Making  Process. 


F 

all.  1971 

Sp 

ring,  1972. 

Ej  S  E2 

E^  4  Cj 

E^  S 

N 

9 

N 

11 

N 

10 

N 

11 

N 

9 

N 

11 

N 

9 

N 

11 

N 

10 

N 
11 

N 

9 

N 

10 

Kruskal-Wal. 

Lis  One-Wa> 

I'  Analysis  of  Variance 

H=ll .430 
Sig.«  .00 

H«  0.447 
lSig.=  NS 

H=ll,430 
Sig.*.001 

H=10.  922 
Sig.sOOl 

H=l,433 
Sig.=  NS 

2.940 
Sig.=  NS 

Binomial  Test  of  Proportions 

p»l  .000 
Sig.»  NS 

pi  .000 
Sig.=NS 

P=1.000 
Sig.*  NS 

P=l  .  000 
Sig.=  NS 

P= 1.000 
.Sig.=  NS 

J'^1 .000 
,''^ig-=  NS 

Fall,  1971 

to  Spring 

.  1972 

El  § 

E2  §  E2 

N 

9 

N 

9 

N 

10 

N 

10 

N 

8 

N 

10 

Kruskal-Wallis  One-Way  Analysis  of  Variance 


H=0.048 
Sig.  =  NS 


H=  1.120 


Sis 


.=  NS 


H=  0.0C( 


Si£ 


,=NS 


Binomial  Test  of  Proportions 


P^l.OOO 

P=  1.000 

P=  0.03C 

Sig.  =  NS 

Sig.=  NS 

Sig.=.05 
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6   I  can  influence, the  goals,  mehhod s , 

and  activities  of  my  orc^anization. 


Fall,  1971 
T 

Spring,  1972 

E^qperiaental  SEDffl 
Experlaental  SBD«2 
Contxol  SED  #1 
Control  SED  «2 
Total 

 4 

N 

68 

X 

5.220 

SD 

1.433 

N 

40 

X 

5.175 

SD 
1.009 

45 

4.577 

1.469 

.40 

4.300 

1.471 

67 

4.731 

1.343 

61 

4.557 

1.554 

61 

4.770 

1.531 

66 

4.545 

1.458 

141 

^07 

IVro  Hay  Analysis  of 
Variance 

Bxperiaental  SBD  #1 
W/Control  SED  #1 
EiqEieriBental  SED  #1 
w/Control  SED  #  2 

P 

Sianif. 

;oi. 

0.356 

NS 

Bpw 

9.060 

.01 

:oi. 

0.520 

NS 

Row 

8.276 

.01 

E]q>eri»ental  SED  *2 

Col. 

1.224 

NS 

w/Control  SED  *1 

Row 

1.013 

NS 

Ejqieriaental  SED  #2 

Col. 

i.807 

NS 

w/Control  SED  #2 

Row 

0.972 

L  NS 
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Analysis  of  Item  .tl,  showed  that  the  MUi  t;raining 
had  no  effect  on  fePB  admin ij5itr atoms'  feeling  of  Involvement 
in  major  organizational  decisions?  neither  El,  E2,  or  CI 
experienced  any  change  between  Fall  1970  and  Spring  1972. 

Prior  to  training,  persons  in  El  felt  more  involved 
than  their  counterparts  in  CI.    If  we  examine  the  three 
post-training*  periods,  El  perceived  its  decisionmaking 
process  to  be  iriore  p^ticipative  than  either  E2  or  CI; 
the  same  differences  occurred  in  Spring  1971#  Pall  1971 
and  Spring  .1972.   .As  far  as  El  itself  was  concerned,  however, 
no  significamt  changes  occurred  during  the  two-* year  period. 

V.o  significant  differences  in  awareness  of  the 
decisionmaking  process  appeared  between  any  of  the  states. 
While  E2  somewhat  decreased  in  aweureness '  over  time  (Fall 
1970  to  Spring  1972),  El  and  CI  remained  essentially  the 
same.  ^. ;  ' 

There  were  no  differences  in  av/areness  of  item  #2, 
Quality 'of  Decisionmaking  in  the  State  Department,  between 
the,  states,  nor  did  any  state  change  over  time. 

, Prior* to  training,  no  differences  Appeared  in 
emphasizing  this  item  between  Bl  and  CI  emd  El  and  E2;  but 
CI  perceived  a  higher  quality  of  decisionmaking  than  E2* 
However,  after  training,  significant  differences  began  to 
appear.    El  attributed  greajber  effectiveness  to  their 
organizational  decisions  than  E2  attributed  to  theirs.  More 
importantly.  El  felt  better  about  their  effectiveness  than 
did  CI  (in  Fall  1971  and  Spring  L972)  while  the  pre-training 
advantage  of  CI  over  E2  became  insignificant  (in  Fall  1971 
and  Spring  1972). 

The  explanation  for  this  development  is'  clear.  For 
the  period  Fall  1970  to  Spring  1972,  neither  Experimental 
State  indicated  a  change  in  the  effectiveness  of  the  deci- 
sionmaking process  as  a  result,  of  training.    The  key  to 
the  relative  positions  of  El,  E2,  and  CI  is  that  CI  dropped 
durihg^.the  same  time.    All  other  things  being  equal,  we 
can  conclude  tl)at,  without  training,  the  Experimental 
States  would  have  undergone  a  similar  decrease  in  perceived 
effectiveness.    Thus  training  had  a  Positive  effect  in  El 
and  E2  because  it  stabilized  the  quality  of  their  decision- 
making process,  while  CI  declined. 

Item  #3,  Influence  of  Planning  on  decisionmaking 
process,  is  an  addition  to  the  Y2  research  design.  Overall, 
El  felt  that  planning  had  more  influence  on  their  decision- 
making process  than  did  E2  or  CI.    Yet  El,  E2,  and  C2  did 
not'  change  in  the  influence  they  perceived  that  planning 
exerted  in  their  organizations. 
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Both  Experimental  States  and  the  Control  State 
showed  similar  levels  of  awareness  of  planning 's  influence , 
although  CI  increased  its  awareness  over  time.  Possibly 
this  may  be  attributed  to  the  fact  that  the  Spring  1972 
interviews  were  conducted  after  CI  entered  AMA  training. 

Because  the  question  was  not  asked  prior  to  train* 
ing,  we  must  be  careful  about  judgments  regarding  the 
impact  of  the  AMA  program  on  the  influence  of  planning  over 
the  El  or  E2  decisionmaking  process.    Nevertheless r  since 
no  significant  change  occurred  in  any  State  between  T3  and 
T4,  we  conclude  that  training  had  virtually  no  effect  on 
El  emd  E2. 

Item  <>4:    Fall  1970  to  Spring  1971 

Item  #4  is  The  people  I  work  with  participate 
appropriately  in  setting  the  goals  of  our  work. For  the 
inmediate  pre/post- training  period,  it  snowea  mixed  effects. 
Training  had  the  effect  of  increasing  participation  in 
decisionmaking  in  El;  it  had  no  effect  in  E2.    Both  Experi- 
mental States  differed  from  the  Control  State  in  the  amount 
of  participation  that  was  possible  within  the  organization. 
There  was  greater  participation  in  El  than  in  the  Control 
State,  and  less  participation  in  E2. 

Item  #4;    Spring  1971  to  Fall  1971 

Both  Experimental  States  experienced  a  drop  in 
participation  and  no  training  effects  during  this  period. 
Persons  in  El  did  feel  more  involved  in  decisionmaking 
than  those  in  CI,  while  E2  felt  less  involved.    But  neither 
El  or  E2  changed  significantly  between  Spring  1971  and  Fall 
1971. 

Item  »4:    Fall  1971  to  Spring  1972 

The  extent  to  which  people  felt  involved  in  goal 
setting  was  not  affected  by  A^!A  training  on  this  item.  CI 
and  C2  reported  significantly  less  participation  than  El, 
while  no  differences  showed  between  E2  and  the  Control 
States.    The  amount  of  participation  continued  to  decline 
in  both  Experimental  States  but  not  enough  to  render  the 
change  significant  in  either  case. 

Item  #4 I    Fall  1970  to  Spring  1972 

The  overall  impact  of  ?\I1A  training  on  the  Experi- 
mental States  can  bo  assessed  by  comparing  the  degree  of 
involvement  felt  by  El  and  E2  in  the  Fall  of  1970,  with  how 
they  felt  in  the  Spring  of  1972.    Analysis  of  El  with  CI 
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indicates  that  training  had  a  positive  effect  in  El;  the 
extent  to  which  persons  in  that  organization  participated 
in  decisions  relating  to  their  work  did  decline  over  the 
four  periods,  but  the  rate  of  decline  was  sharper  in  the 
Control  State  and  significant  differences  existed  between 
them.    Involvement  in  E2  also  declined  but  no  significant 
difference  existed  when  canpared  with  CI.    The  rate  of 
decline  was  greater  in  E2  than  in  CI;  thus  negative  train- 
ing effects  occurred  in  E2. 


Item  #5:    Fall  1970  to  Spring  1971 

Item  #5,  I  am  appropriately  involved  in  decisions 
affecting  rov  worK.  refiPctPd  frha  ««nin  fo1«4-j^»»Wjp« 
previous  item.    A  significant  increase,  a  training  effect, 
was  registered  in  the  way  individuals  were  involved  in 
decisions  in  El.    Training  had  no  effect  in  E2.  Experi- 
mental State  E2  differed  from  the  Control  State  in  the  amount 
of  involvement  individuals  felt  they  had  in  decisions  affect- 
ing their  owrK;  greater  involvement  existed  in  the  Experi- 
mental States.    Organization  E2  also  differed  from  the  . 
Control  Sta^te  by  demonstrating  less  involvement. 

Item  #5;    Spring  1971  to  Fall  1971 

During  this  period  no  change  of  either  greater  or 
lesser  involvement  occurred  in  either  El  or  E2.  Organiza- 
tion El  allowed  significantly  more  individual  participation 
in  decisions  affecting  jobs  than  did  the  Control.    No" differ- 
ence existed  between  E2  and  the  Control  State. 


Item  »5;    Fall  1971  to  Spring  1972 

Organization  El  and  both  Control  States  permitted 
significantly  different  levels  of  involvement;  this  differ- 
ence was  due  to  the  fact  that  persons  in  the  Experimental 
State  felt  more  involved  than  their  counterparts  in  either 
CI  or  C2.    No  difference  existed  between  E2  and  the  Control 
States.    El  and  E2  did  not  change  between  Spring  and  Fall  1971. 

Item  #5;    Fall  1970  to  Spring  1972  , 

For  slightly  different  reasons,  AMA  training  had  a 
positive  effect  in  both  States.    Declining  levels  of  partici- 
pation characterized  El  and  E2  during  this  period;  however, 
neither  declined  as  fast  as  CI.    El  considered  its  decision- 
making process  to  be  more  participative  than  CI,  while  E2 
viewed  itself  as  less  participative  although  not  significantly 
less  so.    E2  was  also  more  stable  than  CI. 
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Item  dfii    Fall  1971  to  Spring  1972 

Item  #6:    I  can  Influence  the  goals,  methods,  and 
activities  of  my  ort^anizationl    Comparison  was  made  on  this 
item  between  two  Experimentai  States  and  two  Control  States 
over  two  points  in  time.    Analysis  reveals  that  all  four 
organizations  believed  they  could  influence  their  goals, 
methods,  and  activities  less  in  the  Spring  of  1972  than  in 
the  Fall  of  1971.    CI  declined  a  little  more  sharply  than 
did  El  but  both  changed  less  than  E2  or  C2.    CI  and  C2  felt 
less  influence  than  El,  but  more  than  E2. 


DATA  SUMI'lARY 
DecisionmeUcing 
Fall,  1970-8prinq,  1972 

Type  of  Data 
Item  CONTENT 

1  Involv<Hnent  in  Decisionmaking 
in  the  State  Department 

2  Quality  of  Decisionmaking  in 
the  State  Department 

3  Influence  of  Planning  on 
decisionmaking  process 


IIIPACT  OF  TRAINING 

Positive       No  Negative 
Effect     Effect  Effect 


El,E2 


E1,E2 


E1,E2 


QUESTIONNAIRE 

4  The  people  I  work  with  parti- 
cinate  appropriately  in  set- 
ting the  goais  of  our  work. 

5  I  am  appropriately  involved 
in  decisions  affecting  my 
work. 

6  I  can  influence  the  goals, 
met;hods,  and  activities  of 
my  organization. 


El 


E1,E2 


E2 


E1,E2 


ERIC 


Analysis  of  the  interview  material  obtained  from 
top  administrators  showed  no  effects  of  training  on  involve- 
ment in  major  organizational  decisions  or  in  planning  s 
influence  on  decisionmaking.    Negative  ^^faining  cffons  are 
indicated  for  the  quality  of  decisionmaking;  both  Bxparimental 
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States  felt  their  decisionmaking  processes  had  grown  less 
effective  between  Fall  1970  and  Spring  1972. 

The  larger  sample  provided  by  the  questionnaire  items 
revealed  mixed  training  effects.    Training  increased  the 
degree  of  participation  that  some  individuals  believed 
others  enjoyed  in  setting  Ei*s  work  goals^  but  decreased 
the  degree  of  this  belief  in  E2.    The  extent  to  which  these 
individuals  in  El  and  E2  felt  involved  in  decisions  affect- 
ing their  own  work  was  enhanced  by  the  AMA  program.  How- 
evQT,  no  training  effects  were  observed  on  their  perceived 
influence  over  the  operations  of  the  entire  department;  it 
remained  about  the  same  before  and  after  training  in  both 
States. 

Overall,  AMA  training  showed  positive  effects  on  two 
items  relating  to  decisionmaking #  and  no  effects  on  three 
others.    Only  in  one  State  on  a  single  item  did  negative 
effects  of  training  appear. 

As  with  the  items  testing  leadership  climate ,  however , 
this  result  did  not  demonstrate  cibsolute  changes  in  a  posi- 
tive direction.    The  gross  scores  on  the  questionnaires 
indicated  that  the  decisionmaking  process  declined  in 
perceived  effectiveness  in  both  States  over  time.    Yet  this 
apparently  straightforward  index  of  training  effects  is 
perhaps  too  simple  and  straightforward.    For  the  best 
comparisons  are  drawn  not  within  one  State,  or  between  two 
Experimental  States,  but  between  Experimental  States  that 
receive  training  and  Control  States  that  lack  training. 

Further,  relative  to  the  Control  States  on  some 
items,  the  descent  was  slower  for  the  Experimental  State; 
on  others  it  was  faster.    This  rationale  is  most  proper  for 
our  analysis,  and  it  is  the  basis  on  which  training  effects 
were  frequently  determined. 


C.    Management  Team  Relations 

Six  items  will  be  utilized  -  two  content  categories 
and  five  questionnaire  items.    Content  categories: 

1.    Promote  cooperative  teamwork 

Interview  Question:    PThat  do  you  think  you 

v;ill  obtain  (have  obtained)  from  the 
Mili  training? 

Range  of  Scale  Possibilities:     (1)  no  value 
to  (7)  maximum  value. 

Points  of  Time:     Tl,  T2,  T3,  T4. 
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States:    El,  E2,  Cl. 

2»    Amount  of  cooperative  teamwork  present 

Interview  Question;    What  are  some  of  the  road- 
blocks to  organizational  change? 

Range  of  scale  Possibilities:    Major  roadblock/ 
always  stops  change,  to  (7)  weak  road- 
block/seldom stops  change. 

Points  of  Time:    Tl,  T2,  T3,  T4. 

States:    El,  E2,  cl. 

The  questionnaire  items  are  as  follows,  and  all  items  are 
from  Points  of  Time  Tl,  T2,  T3,  and  T4. 

3.  My  group  works  hard  to  achieve  its  goals. 

4.  My  work    roup  understands  what  we  are  trying 
to  achie\  J. 

5.  I  feel  my  group  works  well  together. 

I  really  feel  my  inanediate  work  group  is  getting 
things  donel   

'7.    When  differences  arise  in  my  work  group,  we 
have  good  ways  tor  settling  them  ourselveFT 


ERIC 
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Item    1  :  Promote  Cooperative  Team  Work 


Fall.  1970 

Spring.  1971 

Fall.  1971 

Spring.  1972 

^1  5  ^2 

E     8  E 
1    ^  2 

^1      S  h 

^1     5  ^2 

N  N 
8  7 

N  N 
11  10 

N  N 
7  7 

N  N 
8  7 

Kruska 

l-Wallis  One-Waj 

r  Analysis  of  Variance" 

H= 1.477 
SiB.=  NS 

H=  5.565 
Sie.=  .05 

H=0.200 
Sig.=  NS 

H=  10.500 
Sig.=.01 

Binomial  Test  of  Proportions 

P=0.736 
Siji.=  NS 

P=0.528 
.    Sig.=  NS 

P=0.426 
Sie.=  NS 

P=0.180 
Sie.=  NS 

Fall  1970        to              Sorine  1972 

El  8 

E2          §  E^ 

N 
8 

N 
8 

N 
7 

N 
7 

Kruska. 

.-Wall is  One-Way  Analysis  of  Variance 

H=  2.318 

H=  9.800 
Sic.=  .01 

Binomial  Test  of  Proportions 

P=  0.02 
Sig.=  .05 

P=  0.141 
Sig.=  NS 
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Promote  cooperative  teamworks  Item  #1,  was  a  con- 
tent caFegory  applied  only  to  organizations  El  and  E2. 
It  indicated  no  difference  in  emphasis  between  the  States 
prior  to  training.    In  the  post-training  periods,  El 
believed  the  program  promoted  cooperative  teamwork  more 
than  did  E2  on  two  occasions:     Spring  1971  and  Spring  1972, 
with  the  disparity  greater  in  the  latter  period.  Concern- 
ing changes  in  emphasis  over  time.  El  reported  no  signifi- 
cant changes,  while  E2  revealed  an  overall  decline  in  the 
belief  that  the  promotion  of  cooperative  teamwork  was 
due  to  their  AMA  training. 

No  differences  appeared  between  the  States  in  their 
awareness  of  this  issue.    However,  El's  awareness  of  train- 
ing's effects  on  encouraging  cooperative  teamwork  signifi- 
cantly increased  between  Fall  1970  and  Spring  1972. 

The  second  Content  category.  Amount  of  cooperative 
teamwork  present  (one  of  the  possible  roadblocJcs  to  organi- 
zational Change) ,  showed  no  differences  between  the  States 
before  or  after  training.    Assessing  changes  within  the 
States  over  time,  training  showed  an  impact  on  El,  in  that 
El  considered  that  the  degree  of  cooperative  teamwork 
presented  a  smaller  roadblock  after  training  than  before; 
no  measurable  changes  occurred  in  E2  or  Cl. 

Awareness  of  this  variable  was  not  significantly 
different  in  El  as  compared  to  E2  or  Cl  before  the  train- 
ing.    However,  E2  was  more  aware  of  cooperative  teamwork 
as  a  roadblock  than  Cl.     In  the  post-training  periods,  fxve 
points  of  significant  differences  in  awareness  appeared: 
Both  El  and  E2  grew  more  sensitive  to  this  variable  than 
Cl  in  Spring  1971.    Fall  1971  indicated  a  continuing  dif- 
ference, with  E2  more  aware  of  the  presence  of  this  road- 
block than  Cl;  and  E2's  awareness  was  greater  than  either 
El  or  Cl  during  Spring  1972.    There  were  no  changes  in 
awareness  over  time  in  either  El  or  E2,  but  Cl  was  less 
aware  of  cooperative  teamwork  constituting  an  obstacle  xn 
Spring  1972  than  they  had  been  in  Fall  1970. 

Item  »3:     Fall  1970  to  Spring  1971 

Item  #3,  My  group  works  hard  to  achieve  its  goals, 
showed  no  effect  of  training  in  either  Experimental  State. 
In  terms  of  how  strenuously  the  groups  were  described  as 
working,  differences  appeared  between  both  El  and  E2  in 
relation  to  the  Control.    Organization  El  indicated  that 
its  groups  worked  harder  than  did  the  Control's;  E2|s 
work  group  described  itself  as  working  less  energetically 
than  the  Control  group's  description  of  itself. 


337 


Item  #3;    Spring  1971  to  Fall  1971 

No  effects  attributable  to  training  occurred  in 
either  Experimental  State  during  1971.    El  thought  their 
work  group  worked  as  hard  as  those  in  the  Control  State  and 
retained  this  feeling  over  time;  this  relationship  also 
characterized  the  E2/C1  comparison.    Neither  changed,  nor 
did  they  differ  from  the  Control  State. 

Item  #3;    Fall  1971  to  Spring  1972 

El  maintained  its  close  proximity  with  CI  (as  well, 
as  C2)  on  how  it  viewed  goal-directed  behavior.  Neither 
had  revised  its  opinions  up  or  down  since  the  Fall  of  1971. 
However,  training  effects  were  evident  in  E2t  these  effects 
were  negative.    All  States  thought  their  groups  did  not  work 
so  well  in  the  Spring  as  they  had  in  the  Fall;  but  E2's 
decline  was  more  pronounced  than  either  CI  or  C2.    So  the 
effect  of  training  on  E2,  at  least  during  this  period,  was 
to  weaken  this  organization's  efforts  to  attain  educational 
goals . 

Item  #3:    Fall  1970  to  Spring  1972 

When  we  cOTipare  pre-  and  post-training  responses  to 
this  item,  we  find  that  the  AHA  was  able  to  have  an  overall 
positive  influence  over  El  and  E2.    Persons  in  those  States 
reported  significantly  lower  goal-directed  activity  in  the 
Spring  of  1972  then  they  had  before  training  began  in  the 
Fall  of  1970.    However,  as  has  been  the  case  before,  Cl's 
work  had  decreased  at  a  faster  rate.    According  to  the 
logic  of  the  research  design,  we  can  assxime  that  El  and 
E2's  performance  losses  would  have  been  at  least  as  great 
as  Cl's  without  AMA  training,  so  this  represents  a  positive 
effect. 

Item  #4;    Fall  1970  to  Spring  1971 

Item  #4,  a  questionnaire  item.  My  work  group  under- 
stands  what  we  are  trying  to  achieve,  reflected  no  effects 
attributable  to  training  for  either  of  the  Experimental 
States  during  the  Fall  1970  to  Spring  1971  period.  Both 
Experimental  States  differed  from  the  Control  State  in  the 
degree  of  emphasis  given  to  the  item,  however.  Significantly 
greater  understanding  existed  in  El  than  in  the  Control, 
and  significantly  less  ccmprehension  showed  in  E2  than  in 
the  Control. 

Item  »4:    Spring  1971  to  Fall  1971 

E2  had  not  changed  its  comprehension  of  what  was 
expected,  nor  did  they  differ  in  this  regard  from  CI. 
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AMA  training  did  prove  helpful  to  El  during  this  period. 
While  persons  in  that  organization  understood  less  than 
previously  indicated,  this  loss  was  not  as  great  as  the 
Control  State's,  and  a  significant  difference  did  show 
between  them.    This,  therefore,  is  a  positive  effect  of 
training. 

Item  #4:    Fall  1971  to  Spring  1972 

In  El,  training  did  not  increase  work- group  under- 
standing of  the  ends  to  which  their  activities  were  directed. 
Comparing  El  to  CI  and  02,  none  had  undergone  major  changes 
between  the  Pall  of  1971  and  the  Spring  of  1972,  although 
El  remained  significantly  higher  than  CI  in  what  they  knew 
about  their  goals.    E2  felt  less  certain  about  this  organi- 
zational variable  in  Spring  1972  than  had  been  the  case  in 
Fall  1971.    Also,  E2*s  understanding  was  declining  faster 
than  the  awareness  of  either  Control  State. 

Item  #4:    Fall  1970  to  Spring  1972 

Looking  at  the  effects  of  training  from  the  perspec- 
tive of  18  months  (Fall  1970  to  Spring  1972)  reveals  that 
it  had  no  effect  in  El  and  negative  effects  in  E2.  El*s 
understanding  of  group  performance  objectives  did  not 
appreciably  change  after  training  nor  did  they  differ  from 
the  Control  State,  which  had  not  received  as  much  planning 
theory  and  practice  from  the  AMA.    The  negative  impact  on 
E2  is  the  result  of  Cl's  decrease  in  this  area  being  not 
as  great  as  the  decline  which  occurred  in  this  Experimental 
State.    Significant  differences  also  appecired  between  E2 
and  CI;  the  latter  knew  more  about  what  they  were  trying 
to  achieve  than  the  former. 

Item  #5:    Fall  1970  to  Spring  1971 

Positive  effects  can  be  attributed  to  AI'IA  training 
as  far  as  Item  #5,  I  feel       group  vorks  well  together,  is 
concerned .    Neither  El  nor  E2  felt  the  groups  worked  as 
well  together  after  training  as  their  pre-training  expecta- 
tions had  promised.    However,  this  gap  between  promise  and 
performance  was  more  than  equaled  in  the  Control  State, 
resulting  in  positive  effects  in  both  Experimental  States. 

Item  #5:    Spring  1971  to  Fall  1971 

Groups  got  along  together  even  better  in  El  and  E2 
on  a  T2-'x'3  comparison— but  with  negative  results.  Relative 
to  CI,  both  experienced  negative  effects  of  training.  For 
although  the  Experimental  States  had  improved  their  group 
relations,  the  Control  State,  without  any  AMA  training,  had 
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improved  theirs  even  more.    This  slower  rate  of  improve- 
ment in  El  and  E2  can  be  attributed  to  AMA  training. 

Item  ft5;    Fall  1971  to  Spring  1972 

The  positive/negative  effects  cycle  apparent  in  the 
two  previous  periods  did  not  continue  into  this  one.  Signifi- 
cant differences  in  the  levels  of  intra-group  cooperation 
were  recorded  between  El  and  C2  as  well  as  between  E2  and 
CI.    In  each  case  the  relationship  between  the  Experimental 
State  an,:  the  Control  State  differed.    Groups  did  not  work 
as  well  in  C2  as  they  did  in  El,  but  those  in  C2  got  along 
better  than  those  in  E2.    These  differences  were  not  due  to 
either  State  changing  its  work  patterns,  but  merely  to  con- 
tinuing the  old  patterns.    Under  this  condition,  we  can 
attribute  no  effects  to  training. 

Item  #5;    Fall  1970  to  Spring  1972 

A  comparison  of  group  relations  in  each  Experimental 
State  between  Pall  1970  (pre-training)  and  Spring  1972 
(post-training)  provides  an  overall  evaluation  of  the 
effects  of  AMA  training.    The  positive  or  negative  effects 
associated  with  this  item  in  Ei  in  the  past  faded  over 
time.    Consequently,  no  differences  existed  between  El  and 
CI  or  between  El  before  training  and  El  after  training. 
Groups  work  together  less  well  in  E2  than  in  CI;  yet  rela- 
tions in  both  of  these  States  have  deteriorated  since  train- 
ing concluded.    But  assuming  that  E2  would  have  experienced 
a  loss  as  severe  as  CI  had  they  not  gone  through  the  AMA 
team  planning  process,  a  positive  effect  of  training  can 
be  assumed  in  E2,  at  least  on  this  aspect  of  organizational 
cooperation. 

Item  #6;    Fall  1970  to  Spring  1971 

Item  #6,  I  really  feel  my  immediate  work  group  is 
getting  things  done,  refers  to  how  effective  the  smallest 
organizational  units  see  themselves  to  be  in  accomplishing 
what  they  consider  worthwhile  educational  goals.  Apparently, 
training  did  not  affect  the  performance  of  El,  for  when  we 
compare  them  to  CI  no  differences  or  changes  can  be 
ascribed  to  Hamilton.    There  is  a  significant  difference 
and  a  negative  effect  in  E2*    That  E2  was  able  to  accomplish 
less  after  training  than  before  was  significant. 

Item  #6:     Spring  1971  to  Fall  1971 

7\MA  training  had  no  influence  either  way  on  either 
Experimental  State  between  Spring  and  Pall  1971.  Work 
groups  in  El  and  E2  did  not  perform  betuer  as  a  result  of 
training,  although  El  had  much  higher  performance  levels 
^  than  the  Control  State. 
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Item  #6s    Fall  1971  to  Spring  1972 

El  did  not  change  their  effectiveness  during  this 
period.    In  this  they  resembled  CI  and  the  Control  State 
added  for  this  analysis  alone  (C2) .    Such  an  absence  of  any 
differences  over  time  or  between  the  States  makes  it  unlikely 
that  AMA  training  had  any  impact  on  this  Exoerimental  State. 
E2  was  not  so  fortunate;  training  hurt  their  effectiveness 
during  this  year.    As  for  accomplishment,  E2  did  less 
between  Pall  1971  and  Spring  1972.    This  downward  trend  was 
the  same  in  01  but  the  rate  of  descent  was  slower  even 
though  CI  lacked  as  much  exposure  to  the  AMA  planning  process. 

Item  #6;    Fall  1970  to  Spring  1972 

No  differences  existed  between  CI  and  either  Experi- 
mental State.    Performance  at  canparable  levels  was  shown 
in  both  sets  of  comparisons o    Nevertheless,  the  accomplish- 
ments of  these  organizations  betv/een  Fall  1970  and  Spring 
1972  had  changed;  in  all  cases  the  change  represented  a  net 
loss  In  effectiveness.    But  while  absolute  levels  declined 
in  the  States,  the  speed  at  which  they  lost  their  ability 
to  get  things  done  differed.    The  CI  comparison  demonstrates 
a  positive  effect  of  training  in  El  and  a  negative  effect 
in  E2.    Training  enabled  El  to  slow  its  descent  in  relation 
to  CI,  but  training  influenced  E2  to  accomplish  less  at  a 
faster  rate  than  the  Control  State. 


Item  #7;    Fall  1970  to  Spring  1971 


Item  #7,  When  differences  arise  in  my  work  group/ 
WG  have  good  ways  for  settling  them  ourselves,  is  an  index 
of  group  effectiveness  in  resolving  conflict.    E2  is  less 
proficient  at  settling  conflict  than  the  Control  State, 
while  El,  E2  and  CI  all  were  better  at  settling  disagreement 
before  training.    With  training,  however.  El  and  E2  got 
worse  at  a  slower  rate  than  CI  did  without  training.  Both 
Experimental  States  were  thus  affected  positively  by  their 
AMA  experience. 


Item  #7;    Spring  1971  to  Fall  1971 

No  changes  occurred  in  1S71  in  the  Experimental 
States'  ability  to  resolve  conflict  within  their  work  group. 
They  were  also  not  significantly  different  than  the  Control 
State.    AMA,  therefore,  had  done  nothing  for  (or  to)  any 
State  during  this  period. 
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Item  #7:    Fall  1971  to  Spring  1972 

Hero  again^  no  changes  appeared  over  time  nor  any 
differences  between  the  States.    Even  the  arlr^.ition  of  the 
second  Control  State  does  not  alter  the  lack  of  any  effect 
of  AHA  training  in  El  and  E2  between  the  Fall  of  1971  and 
the  following  Spring, 


Item  #7;     Fall  1970  to  Spring  1972 

We  now  compare  whether  the  Experimental  States  were 
able  to  resolve  group  disagreement  better  before  AIIA  train- 
ing or  over  a  year  after  training.    As  is  the  case  in  many 
programs  of  this  type^  the  positive  effects  observed  in  El 
immediately  after  training  had  dissipated  completely  by 
Spring  1972.    But  E2  seemed  to  be  helped  by  the  program, 
since  the  Control  State's  skill  in  dealing  with  conflict 
decreased  faster  than  did  E2's. 


DATA  SUMMARY 
Management  Team  Relations 
Fall,  1970  -  Spring,  1972 

If^PACT  OF  TRAINING  

Type  of  Data  Z     TT"!  i;  7! 
  Positive         No  Negative 

Item  CONTENT  Effect       Effect  Effect 

1  Promote  cooperative  teamwork  El  E2 

2  ;\mount  of  cooperative  team-- 
work El  E2 

QUESTIONNAIRE 

3  My  group  works  hard  to 

achieve  its  goals.  E1,E2 

4  My  work  group  understands 
what  we  are  trying  to 

achieve.  El  E2 

5  I  feel  my  group  works  well 

together,  E2  El 

6  I  really  feel  my  immediate 
work  group  is  getting 

things  done.  El  E2 

7  When  differences  arise  in 
my  work  group,  v;e  have  good 
ways  for  settling  them 

ourselves.  e2  El 
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Cooperative  teamwork  au  essential  to  the  team  plan- 
ning process  was  treated  in  two  ways  by  the  content  analysis. 
It  was  considered  as  a  possible  benefit  of  AMA  training, 
and  as  a  potential  obstacle  to  organizational  change. 
El  received  nearly  what  it  expected  before  training  by 
way  of  benefit,  while  E2  benefited  much  less  than  antici- 
pated.   As  a  roadblock,  cooperative  teamwork  remained  at 
similar  pre-  and  post-training  levels  in  E2,  and  diminished 
in  El. 

The  questionnaire  items  associated  with  group  effec- 
tiveness recorded  across-the-board  reductions  between 
Fall  1970  and  Spring  1972.    Nevertheless,  because  of  the 
need  to  compare  these  Experimental  State  changes  with 
developments  in  the  Control  States  during  the  same 
period,  negative  effects  were  assigned  on  only  two  items. 

Mixed  training  effects  were  associated  with  the 
smooth  functioning  of  work  groups  in  both  States.  Groups 
understood  less  about  expected  results  in  E2;  they  improved 
techniques  for  settling  conflict  in  E2;  they  acc<^plished 
more  in  El  and  less  in  E2,  and  worked  together  better  in 
E2.    The  £malysis  showed  no  other  effects « 

E2*8  reaction  to  AMA  training  appears  rather 
unstable,  as  evinced  by  the  numerous  positive  and  negative 
effects  attributed  to  training,  as  well  as  by  the  scarcity 
of  E2  items  on  which  training  had  no  effect.  Training 
effects  were  minimal  in  E2;  their  management  team  rela- 
tions were  largely  untouched  by  the  training  process. 
The  AMA  team  planning  process  did  not  create  a  team  in 
Ei?  it  merely  worked  with  one  that  already  existed^ 


Section  3;    LEA  Attitudinal  Data 

The  intervening  variables  of  the  local  education 
agencies  reflect  their  internal  health.    These  variables 
help  deteiiaine  if  the  procedures  and  principles  of  the  A!4A 
team  planning  process  will  be  translated  into  meaningful 
changes  and  improved  educational  products. 

The  approach  taken  in  this  section  resembles  the 
one  we  used  in  the  SED's,  where  data  was  organized  into 
comparable  categories: 

A*    Leadership  Climate 

B •  Decisionmaking 

C.    Management  Team  Relations 
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Data  was  gathered  from  participating  LEA's  through 
the  use  of  questionnaires  only.    In  displaying  question-- 
naire  data  within  each  category,  two  ccxnmonts  will  be 
provided: 

1.  Questionnaire  Item;    The  statement  in  the 
questionnaire  to  which  a  response  was  given  on  a  scale  of 
(1)  not  at  all  to  (7)  very  often. 

2.  Points  of  Time:    For  each  item,  the  time  at 
which  it  was  administered  will  be  indicated  (Tl  =  Fall 
1970;  T2  =  Spring  1971;  T3  =  Fall  1971;  T4  =  Spring  1972). 

The  questionnaire  items  were  administered  in  the 
four  Bxperimental  groups  during  all  time  periods.  In 
addition,  two  Control  groups  were  added  for  the  T3  and  T/5 
cc^nparisons.    Since  every  questionnaire  item  was  asked, 
at  varying  times,  to  the  four  Experimental  groups  and  two 
Control  groups,  it  will  not  be  necessary  to  specify  the 
LEA*s  on  each  item. 

Results  of  training  were  usually  determined  on  the 
basis  of  ai  T1*T4  comparison.    In  some  instances,  however, 
it  was  considered  appropriate  to  attribute  effects  after 
T3-T4  comparisons.    While  offering  a  shorter  time  span, 
these  Fall  1971  to  Spring  1972  ccmiparisons  were  done  with 
Control  groups  present.    The  T1-T4  comparisons  lack  Control 
groups.    The  only  exception  to  this  general  rule  involves 
those  items  that  were  designed  to  measure  attitudes  toward 
AMA's  role  in  the  training.    Since  questions  about  the 
AMA  had  limited  relevance  to  those  organizations  which  did 
not  participate  in  the  program,  the  Control  LEA*s  were  not 
askec!  to  answer  such  questions. 

Because  of  the  procedural  and  methodological  safe- 
guards built  into  this  research  design,  we  have  almost 
equal  confidence  in  the  conclusions  drawn  from  both  sets 
of  evidence:    T1-T4  and  T3-T4. 


A.    Leadership  Climate 

Six  (6)  questionnaire  items  were  selected  to  assess 
the  leadership  climate  within  the  LEA's: 

1.  Based  on  information  I  have  received  from  my 
DOSS,  I  know  if  I  am  measuring  up  In  my  job. 

Points  of  Time:  Tl,  T2,  T3,  T4  (The  other 
items  in  this  category  were  seunpled 
at  the  same  points  in  time.) 

2.  My  manager  encourages  and  supports  innovation. 
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3.  Higher  manag^ent's  reactioigto  the  problems 
which  reach  them  are  fair> 

4.  My  manager  knows  and  understands  the  problems 
I  face>  ^ 

5.  My  manager  recognizes  when  a  problem  Is 
developing  and  goes  something  constructive 
about  lt> ~ 

6.  My  manager  shows  confidence  and  trust  in  me. 


1 
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Item  #1:    Fall  1970  to  Spring  1971 

Item  #1  is  Based  on  information  I  have  received 
from  my  boss ^  I  know  if  I  am  measuring  up  in  my  job^  No 
significant  differences  existed  between  EL1/EL2  or  EL3/ 
EL4  although  all  four  demonstrated  less  feedback  on  job 
performance  between  Pall  1971  and  Spring  1971.  Introduc- 
tion of  the  ANA  change  strategies  to  ELI  or  EL2  cannot  be 
held  to  have  affected  either  organization,  given  their 
slight  shifts  over  time.    However,  recalling  the  deleteri-* 
ous  effect  that  lack  of  a  control  group  exerts  on  our 
findings,  we  still  sense  a  negative  training  effect  in 
other  Experimental  LEA's.    Decreased  information  flow  was 
sufficient  to  warrant  such  a  conclusion  in  EL3  and  EL4. 

Item  #1:    Spring  1971  to  Fall  1971 

EL3  and  EL4  reversed  their  downward  trend  'during 
this  post-training  period;  information  on  how  well  members 
were  performing  flowed  better  in  the  Fall  of  1971  than  in 
the  Spring «    Since  these  more  recent  increases  were  not 
very  large,  however,  it  is  impossible  to  give  AI!A  great 
credit  for  them.    The  AMA  exerted  another  positive  impact 
on  ELI  and  EL2,  since  both  groups  reported  significant 
increases  with  no  measurable  disparity  between  them.  We 
can  ascribe  positive  training  effects  to  both  organizations 
during  this  period. 

Item  »1:    Fall  1971  to  Spring  1972 

Again,  we  can  have  more  confidence  about  our  analysis 
here  due  to  the  addition  of  Control  LEA's  against  v;hich  to 
compare  Experimental  LEA  progress.    ELI  offers  considerably 
more  data  on  job  performance  than  EL2  or  CLl.    But  compar- 
ing the  Experimental  groups  against  each  other  and  against 
their  respective  controls,  no  definite  effects  of  the  AMA 
experience  appear  between  Fall  1971  and  Spring  1972. 

Item  #1:    Fall  1970  to  Spring  1972 

A  review  of  possible  changes  over  this  18-month 
period  reveals  that  bosses  relayed  slightly  more  informa- 
tion after  training  in  ELI  and  EL2,  but  slightly  less  in 
EL3  and  EL4.    These  small  gains  and  losses  were  in  all 
cases  insignificant.    Due  to  this  statistical  stability, 
no  training  effects  were  measurable. 

Item  #2:    Fall  1970  to  Spring  1971 

Item  #2  is  My  manager  encourages  and  supports  inno- 
vation.   During  the  Fall  1970  to  Spring  1971  period,  three 
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Experimental  LEA's  (ELI,  EL3  and  EL4)  showed  declining 
support  for  new  ideas  and  change,  while  the  fourth  (EL2) 
increased  slightly.    Only  in  the  case  of  ELI  was  the  down- 
ward shift  significant;  it  reflected  a  negative  effect 
of  training.    No  other  LEA  reported  a  loss  of  such 
magnitude « 

Item  »2;    Spring  1971  to  Fall  1971 

The  negative  training  effects  observed  in  ELI  were 
temporary;  this  Experimental  LEA  showed  a  much  more  favor- 
able impact  of  training  T2-T3.    The  degree  to  which  their 
managers  encouraged  innovation  rose  considerably  and  this 
was  interpreted  as  a  positive  training  effects    EL2  also 
enhanced  their  supportive  behavior  but  not  enough  to  be 
significant  nor  to  open  any  gap  with  ELI.  Similar 
increases  turned  up  in  EL3  and  EL4,  but  these  were  incon- 
sequential; neither  Experimental  LEA  differed  from  the 
other  in  a  major  way.    Overall,  therefore,  the  period 
Spring  to  Fall  1971  brought  positive  effects  to  ELI  and 
no  effects  to  EL2,  EL3,  or  EL4. 

Item  »2:    Fall  1971  to  Spring  1972 

Encouragement  and  support  for  innovation  fell  in 
all  Experimental  groups  over  time,  but  rone  declined 
enough  to  label  this  fall  significant.    A  further  compari- 
son with  the  Control  LEA*8  reinforces  this  conclusion. 
While  ELI  showed  substantially  more  innovative  activity 
than  CLl,  and  CL2'6  managers  favored  change  more  than 
either  EL3  or  EL4    managers,  these  variations  were  stable 
over  time — all  Experimental  LEA's  felt  nearly  the  same  In 
both  periods.    Without  any  sizable  modifications,  it  was 
not  possible  to  attribute  any  training  effects  to  AMA  in 
T3-T4. 

Item  #2:    Fall  1970  to  Spring  1972 

Taking  a  longitudinal  view,  we  found  that  all 
managers  acted  similarly  toward  innovation  before  and 
after  training.    Any  visible  veuriations  up  or  down  or 
between  groups  proved  to  be  inconsequential.    We  thus  con- 
cluded that  the  AMA  pilot  program  had  no  effect  on  the 
degree  to  which  managers  in  the  LEA^s  supported  innovative 
activities. 

Item  #3;    Fall  1970  to  Spring  1971 

Item  #3  concerns  how  superiors  respond  to  problems 
brought  to  their  attention:    Higher  management ' s  reactions 
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to  the  problems  which  reach  thorn  are  fair.  Negative 
training  effects  occurred  in  EL2,  who  felt  that  reactions 
of  their  superiors  were  more  unfair  after  training  than 
before.    In  this  they  were  substantially  below  the  other 
Experimental  LEA  in  that  State/  ELI.    No  effects  were 
determined  in  EL3  or  EL4.    All  LEA's^  except  EL3^  met 
with  less  favorable  management  responses  between  Fall 
1970  and  Spring  1971. 

Item  #3;    Spring  1971  to  Fall  1971 

There  were  no  impressive  changes  on  this  organiza- 
tional variable  in  EL3  or  EL4  in  1971,  although  both 
improved  moderately.    The  real  changes  occurred  in  EL2; 
higher  management's  reactions  were  much  fairer  in  the  Fall 
than  they  had  been  in  the  Spring.    This  transforation  was 
definitely  a  result    of  AMA  training.    El  also  enhanced 
their  communication  system  but  not  enough  to  attribute 
the  improvement  to  the  team  planning  process  learned  at 
Hamilton. 

Item  #3;    Fall  1971  to  Spring  1972 

All  Experimental  LEA* s  declined  during  this  period 
although  in  no  instance  were  reductions  significant. 
Differences  were  created  by  the  varied  rates  of  decline r 
however.    EL2  thought  their  managers  responded  to  them 
more  unfairly  than  did  ELI,  and  this  difference  was  con- 
siderable.   When  compared  to  the  Control  LEA  added  to 
strengthen  the  Y2  research  design,  EL2  also  showed  at  a 
distinct  disadvantage.    No  other  difference  was  observed 
between  any  Experimental  LEA  or  between  any  Experimental 
and  Control  LEA's.  • 

Item  #3;    Fall  1970  to  Spring  1972 

The  crucial  T1-T4  comparison  revealed  no  training 
effects.    While  ELI  continued  to  feel  better  about  their 
managerial  relations  than  EL2,  no  changes  over  time  were 
involved  in  this  difference.    Nothing  distinguished  EL3 
from  EL4  although  EL3  increased  slightly  while  EL4  showed 
a  modest  decline.    Overall,  therefore,  in  all  LEAs  the 
AMA  program  did  not  influence  higher  management's  reac- 
tions  to  problems  that  reached  them. 

Item  #4;    Fall  1970  to  Spring  1971 

Item  #4,  My  manager  knows  and  understands  the  prob- 
lems I  face,  is  another  t actor  forming  a  suitable  leader- 
ship climate.    Between  Fall  1970  and  Spring  1971,  management 
understanding  in  the  LEA^s  declined;  most  of  all  in  ELl. 
In  fact  the  decreases  in  that  organization  were  so 
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pronounced  when  compared  to  FL2  that  the  AMA  was  held 
accountable  for  negative  training  effects  in  ELI,  There 
existed  no  further  change  over  time  in  any  LEA,  or  sub-- 
stantial  differences  between  them.     Interpretation  of 
this  data  depended  upon  the  four  Experimental  LEA's 
alone #  without  benefit  of  a  Control' LEA;  so  we  were 
especially  careful  of  our  attributions. 

Item  j?4:    Spring  1971  to  Fall  1971 

No  significant  differences  between  the  Experimental 
LEA'S  were  reported  during  1971,    With  two  exceptions, 
no  shifts  appeared  in  these  agencies  in  how  managers 
viewed  problems.    The  exceptions  were  ELI  and  EL2,  While 
the  managerial  empathy  aspect  of  their  leadership  climate 
had  declined  T1-T2,  it  rose  appreciably  T2-T3,    ELI  v;as 
able  to  credit  training  with  positive  effects;  positive 
effects  followed  in  EL2,    Both  groups  also  exhibited  a 
growing  consensus  favoring  this  climatic  improvement. 

Item  #4:    Fall  1971  to  Spring  1972 

The  addition  of  the  Control  LEA's  strengthen  our 
findings  here.    Two  of  these  Control  group  comparisons 
provided  data  on  training  effects.     In  one  case  (ELl/CLl) 
these  effects  were  positive,  since  the  Experimental  LEA's 
managerial  knowledge  of  subordinate  difficulties  increased 
again,  while  the  same  factor  declined  in  the  pontrol  LEA's 
The  other  instance  showed  EL4  and  CL2  decline;  but  since 
the  LEA  which  had  been  trained  fell  more  severely  than 
the  LEA  which  had  not,  negative  effects  were  deteirmined 
in  EL4, 


Item  #4:    Fall  1970  to  Spring  1972 

The  Experimental  LEA^i*  feelings  on  this  item  before 
training  (Fall  1970)  compared  with  how  they  felt  after 
training  (Spring  1972)  evi.iced  the  durability  of  training 
effects.    It  appears  in  this  case  that  these  effects 
lacked  durability.    For  whatever  the  effects  had  been 
earlier,  in  this  most  important  comparison  they  had 
vanished.    We  found  no  measurable  changes  or  differences, 
and  thus  no  training  effects  were  assigned. 

Item  #5:    Fall  1970  to  Spring  1971 

What  top  administrators  do  about  problems  develop- 
ing in  their  organizations  is  the  subject  of  Item  #5,  My 
manager  recognizes  when  a  problem  is  developing  and  does 
something  constructive  about  iti    From  the  pre-  to  immedi- 
ate post- training  periods,  all  LEA's  experienced  negative 
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training  effects^    Recognition  of  emerging  issues  fell 
substantially  as  a  result  of  the  AilA  program,  Ther^ 
were  no  major  differences  between  FL1/EL2  or  EL3/EL  -  at 
this  time. 

Item  #5;    Spring  1971  to  Fall  1971 

ELS  and  EL4  met  with  no  improvements  in  the  IP 71 
post-training  periods.    Their  managements'  respo;^  ?es  to 
problems  stabilized  at  about  the  same  level  as  before. 
However,  positive  training  effects  occurred  in  ELl.  Both 
ELl  and  EL2  gained  considerably  on  these  items  between 
Spring  and  Fall,  but  only  ELl  was  assigned  positive 
effects  because  of  its  steeper  rise  and  because  persons 
in  ELl  showed  tremendously  increased  consensus  reflecting 
the  higher  opinion  of  their  leadership. 

Item  f:St    Fall  1S71  to  Spring  1072 

No  difference  showed  in  any  Experimental  LEA  or 
between  the  Experimental  LEA's  and  the  Control  LEA"s. 
Furthermore,  most  of  the  management  teeims  that  were 
trained  were  not  viewed  any  differently  in  the  Spring  of 
1972  than  they  had  been  in  the  Fall  of  1971  Nevertheless, 
a  minority  repoit  cropped  up  here;  ELS  and  EL4  did  change. 
They  reduced  their  constructive  approach  to  problems,  in 
declines  significant  enough  to  have  occurred  as  a  result 
of  AIIA  training. 

Item  #5:     Fall  1970  to  Spring  1972 

The  negative  change  in  EL3  and  EL4  carried  over  into 
the  T1-T4  comparison..  An  overall  assessment  of  the  impact 
of  the  team  planning  process  revealed  that  it  adversely 
affected  the  leadership  climate  in  both  organizations. 
EL2  had  slightly  greater  success  on  this  item  while  ELl 
had  a  small  decline,  but  neither  group* s  change  was 
significant.    Thus  no  training  effects  v;ere  attributed 
to  the  AMA  in  ELl  or  EL2. 

Item  #6:    Fall  1970  to  Spring  1971 

My  manager  shows  confidence  and  trust  in  me  is  Item 
06  of  this  analysis,  which  shov/s  LEA  declines  in  this  area 
between  Fall  1970  and  Spring  1971.     As  far  as  EL3  and  EL4 
were  concerned,  the  loss  of  confidence  was  not  sizable 
enough  to  be  attributed  to  the  AMA.     However,  ELl  and  EL2'8 
drop  was  sufficient  for  the  program  to  be  held  responsible. 
In  addition,  agreement  among  persons  in  these  organizations 
was  fragmented  during  this  time,  as  indicated  by  the  in- 
creased standard  deviation. 
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Item  i}6i    Spring  1971  to  Fall  1971 

This  decline  was  reversed  in  1971.    Both  ELI  and  EL2 
showed  favorable  upward  modification  on  this  variable. 
Confidence  and  trust  were  considerably  improved  as  was 
organizational  consensus  about  these  favorable  elements  of 
leadership  climate.    Although  small  positive  shifts 
appeared  in  EL3  and  EL4,  these  were  minimal  and  reflected 
a  stabilization  of  earlier  beliefs. 


Item  #6;    Fall  1971  to  Spring  1972 

ELI  became  different  from  EL2  in  T3-T4.    Both  Experi- 
mental leas'  management  relations  grew  more  strained;  how- 
ever, E2  developed  a  significantly  more  hostile  climate. 
There  were  no  training  effects  since  neither  group  had 
changed  much  over  time.    A  comparison  of  EL3  with  EL4  also 
indicated  that  the  team  planning  process  had  not  affected 
these  organizations.    Neither  underwent  major  changes  or 
developed  significant  differences  from  the  other.  Consid- 
eration of  the  Control  LEA's  reinforces  this  "no  effects" 
judgment. 


Item  #68    Fall  1970  to  Spring  1972 

If  we  take  the  longest  possible  view  of  training 
effects  (between  Pall  1970  and  Spring  1972)  none  occurred 
in  the  LEA's  that  participated  in  the  AMA  program.  EL2 
showed  slightly  less  confidence  and  trust  than  ELI  before 
training,  and  substantially  less  a  year  after  training. 
Nonetheless  they  did  not  demonstrate  major  changes  over 
time.    EL3  and  EL4  had  no  differences  batween  them  nor 
had  they  much  shifted  these  opinions  between  Tl  and  T4. 


357 


DATA  SUMMJiRY 
Leadership  Climate 
Fall,  1970  '  Spring,  1972 

Type  of  Data 
Item  QUESTIONNAIRE 

1  Based  on  information  I  have 
received  from  my  boss,  I  know 
if  I  am  measuring  up  in  my  job. 

2  My  manager  encourages  and  sup- 
ports innovation. 

3  Higher  management*  s  reactions 
to  the  problems  which  reach 
them  are  fair. 

4  My  manager  knows  and  under- 
stands the  problems  I  face. 

5  My  manager  recognizes  when  a 
problem  is  developing  and 
does  something  constructive 
cibout  it. 

6  My  manager  shows  confidence 
and  trust  in  me. 


IIIPACT  OF  TRAINING 


Positive 
Effect 


No  Negative 
Effect  Effect 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2,  EL3,EL4 

EL1,EL2, 
EL3,EL4 


Six  items  wore  designed  to  measure  the  impact  of 
7\MA  training  on  the  leadership  slimate  in  tne  LEA's.  Taking 
the  longest  possible  view  (pre-training,  Fall  1970,  to 
Spring  1972) ,  we  find  that  with  two  exceptions  no  signifi- 
cant effects  on  the  relations  between  managers  and  their 
subordinates  were  shown. 

Two  exceptions  to  this  general  pattern  appear  on 
Item  ^5,  on  which  EL3  and  EL4  experienced  negative  effects 
of  training  concerning  the  extent  to  which  managers  seemed 
able  to  recognize  and  resolve  problems. 

Looking  at  the  individual  LEA's,  we  see  a  common 
pattern  in  which  ELI  generally  believed  it  had  a  more 
positive  leadership  climate  than  any  of  the  other  LEA's^ 
although  this  feeling  was  not  statistically  significant. 
Over  time,  all  the  school  districts  experienced  a 
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stabilization  effect  and  showed  approximately  the  same 
attitudes  toward  their  leadership  after  training,  as  they 
had  held  before  it.    The  training  program  with  the  AMA 
did  not  affect  attitudes  concerning  this  category  of 
variables,  with  the  two  exceptions  noted. 

This  analysis  clearly  indicates  that  ATIA  training 
does  not  influence  subordinates'  attitudes  toward  managers. 
This  may  be  the  direct  result  of  the  fact  that  the  managers 
do  not  act  differently  before  and  after  AMA  training. 


S.  Decisionmaking 

The  analysis  of  decisionmaking  will  be  based  on 
three  questionnaire  items: 

1.  The  people  I  work  with  participate  appro- 
priatexy  xn  setting  the  goals  of  our  work. 

Points  of  Time:    Tl,  T2,  T3.  T4 

2.  I  am  appropriately  involved  in  decisions 
affecting  my  work. 

Points  of  Time:    Tl,  T2,  T3,  T4 

3.  I  jan  influence  the  goals,  methods,  and 
gjtivitles  o^  my  organization. 


Poin'vf  of  Time:    T3,  T4 
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Item     3  I  can  influence  the  goals,  methods. 

and  activities  of  mv  organization. 


Fall,  19-^1 
T3 

Spring,  1972 
T4 

experimental  LcA  t^i 
Experimental  LEA  #2 
Experimental  LEA  'r3 
Experimental  LEA  #4 
control  LcA  If L 
Control  LEA  #2 

1  0131 

N 

X 

SD 

N 

X 

SD 

32 

31 

5.774 

0.804 

31 

5.322 

1.221 

32 

5.375 

1.263 

24 

5.125 

1.483 

23 

5.000 

1.087 

29 

5.517 

1.121 

27 

5 . 000 

1.664 

s  ^fifi 

n .  9fi7 

9n 

5  450 

5.13ft 

1.312 

33 

5.363 

1 .167 

182 

166 

Two  Way  Analysis  of 
Variance 

T3  5 

Experimental  LEA#1W/ 
txperimental  LEA#2 

F 

Signif. 

0.040 

NS 

Row 

4.766 

.05 

experimental  LEA#3W/ 
Experimental  LEA#4 

1.4;? 

.  NS 

Row 

0.528 

NS 

Experimental  LEA#1W/ 
pontrol  LEA  #i 

Col. 

n.ofifi 

NS 

.Row 

1.997 

NS 

Experimental  LEA#2W/ 
Control  LEA  #1 

Col. 

0.020 

NS 

Row 

0.515 

NS 

Experimental  LEA#3W/ 
Control  LEA  #2  _ 

Col 

0.043 

NS 

Row 

0.617 

NS 

Experimental  LEA#4W/ 
Control  LEA  #2 

Col 

0.378 

Row 

0.000 

NS 

ERIC 
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Item  #1;     Fall  1970  to  Spring  1971 

Item  #lr  The  people  I  work  with  participate  appro- 
priately in  setting  the  goals  of  our  work^  indicates 
felt  involvement  in  organizational  decisionmaking . 
Measurement  in  Fall  1970  (before  training)  and  again  in 
Spring  1971  (immediately  after  training)   indicates  ELI, 
EL2,  and  EL4  considered  that  their  role  in  setting  goals 
diminished  after  training.     On  the  other  hand,  ELS  sensed 
a  minor  enlargement  of  their  involvement,  but  not  a 
significant  one.    Neither  was  the  decline  endured  by  EL3 
significant.     However,  the  reductions  which  transpired  in 
ELI  and  EL2  were  substantial  and  comparable  in  each 
organization.    Although  the  lack  of  a  control  group  must 
be  considered  in  appraising  training  effects,  it  is  still 
clear  that  decreased  levels  of  participation  in  ELI  and 
EL2  were  negative  impacts  of  the  hMA  program. 

Item  #1;     Spring  1971  to  Fall  1971 

Comparing  T2-T3  responses  to  this  item  reveals 
across-the-board  positive  effects  of  training.  Every 
Experimental  LEA  increased  their  involvement  in  estab- 
lishing work  goals,  and  in  every  case  the  increase. was 
major.     ELI  showed  much  greater  participation  than  EL2; 
while  EL3  and  EL4  were  about  the  saJtie. 

Item  #1:     Fall  1971  to  Spring  1972 

What  occurred  in  T1-T2  reoccurred  in  T3-T4  on  a 
larger  scaled    Not  only  did  EL1/EL2  reduce  participation 
in  decisions  on  goals,  but  this  downward  movement  took 
place  in  EL3/EL4.     Losses  in  the  former  wei-e  minimal 
while  those  in  the  latter  were  sizable.     However,  compar- 
ing both  sets  of  LEA's  to  their  respective  Controls 
indicates  no  effects  of  training.     No  significant  changes 
over  time  nor  any  significant  differences  between  the 
Experimental  and  Control  LEA's  were  found. 

Item  #1;     Fall  1970  to  Spring  1972 

Small  increases  in  appropriate  partio .paticn  in  goal 
setting  were  experienced  in  ELI.     Comparable  increases 
appeared  in  EL2  and  EL3.     EL4's  involvement  weakaned  slight 
ly.     None  of  these  changes  was  significant  from  Fall  1970 
to  Spring  1972.     And  since  this  long  view  is  the  most 
crucial  comparison,  we  cannot  hold  training  accountable 
for  any  effects  in  any  Experimental  LEA's. 
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Item  #2;     Fall  1970  to  Spring  1971 

Item  #2,  I  am  appropriately  involved  in  decisions 
affecting  my  work/  is  an  audit  of  the  Experimental  LEA*s  (4) 
over  two  points  of  time:    pro-  and  immediately  post-train- 
ing.    In  EL3  and  EL4/  AMA  cannot  be  held  responsible  for 
their  losses  in  involvement  since  these  changes  were  not 
at  significant  levels;  these  organizations  also  showed 
comparable  levels  of  participation.    Yet  ELI  and  EL2  were 
significantly  less  participative  after  training  than  they 
had  been  before.    Due  to  these  considerable  reductions 
(with  no  measurable  variation  between  ELI  and  EL2)/  the 
AMA  must  be  assigned  negative  training  effects. 

Item  #2:     Spring  1971  to  Fall  1971 

Involvement  in  the  LEA's  was  enhanced  by  their 
introduction  to  the  team  planning  process  during  this 
period.    Considerable  increases  were  reported  by  them. 
ELI  not  only  felt  the  greatest  influence  over  their  deci- 
sionmaking process  but  also  reached  closer  consensus  on 
this  point  (as  shown  by  a  drop  in  the  SD  column — standard 
deviation) . 

Item  #2;    Fall  1971  to  Spring  1972 

EL3  and  EL4  sensed  more  participation  in  the  Fall 
of  1971  than  they  did  in  the  following  Spring.    While  this 
decrease  was  significant  when  each  was  compared  to  the 
otlier,  it  was  insignificant  when  Control  LEA  (ELI)  entered 
the  analysis.    Consequent ly,  the  fact  that  T3-T4  showed 
diminishing  involvement  in  work-related  decisions  in  these 
lea's  is  not  the  result  of  the  AMA  program.    During  this 
period*  ELI  and  EL2/  while  consistent  in  their  different 
levels  of  involvement,  were  also  stable  over  time;  no 
training  effects  appeared  in  them,  either. 

Item  ^2;    Fall  1970  to  Spring  1972 

The  most  valuable  perspective  is  longitudinal.  We 
examine  felt  participation  before  and  after  training  to 
assess  what  effect,  if  any,  ATIA  exerted  on  the  decision- 
making processes  a  full  year  after  training.    There  does 
not  appear  to  be  much  difference  in  any  group's  involve- 
ment between  Fall  1970  and  Spring  1972.    Any  superficial 
differences  proved  statistically  insignificant.    What  the 
LEA*s  did  at  Hamilton  seemed  to  have  no  effect  on  this 
aspect  of  their  decisionmaking  processes. 
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Item  #3;    Fall  1971  to  Spring  1972 

Item  #3,  I  can  influence  the  goals ^  mGthods#  and 
activities  of  my  organization^  was  ashed  only > in  Y2,  Any 
conclusions  drawn  from  this  item  must  be  handled  with 
caution.    An  analysis  of  the  data  Indicates  no  significant 
differences  over  time  between  the  LEA's  and  the  Control 
groups.    Only  one  significant  difference  appears  in  a 
comparison  of  two  LEA's:    ELI  and  EL2  differed  signifi- 
cantly? personnel  in  ELI  saw  themselves  as  more  influen- 
tial on  the  goals  of  the  organization  than  personnel  in 
EL2.    However,  because  no  significant  change  occurred 
over  time  in  either  LEA,  no  training  effects  can  be 
ascribed. 


DATA  SUMMARY 
Decisionmaking 
ran,  1970  -  sprlnc,,  1972 


Type  of  Data  -i..            „  ^. 
  Positive  No  Negative 

Item  QUESTIONNAIRE  Effect       Effect  Effect 

1  The  people  I  work  with 
participate  appropriately 

in  setting  the  goals  of  EL1,EL2, 

our  worrk.  EL3,EL4 

2  I  am  appropriately  in- 
volved in  decisions  affect-  EL1,EL2, 
ing  my  work.  EL3,EL4 

3  I  can  influence  the  goals, 

methods,  and  activities  of  EL1,EL2, 

my  organization.  EL3,EL4 


An  analysis  of  these  three  items  about  the  decision- 
meiking  climate  in  the  LEA's  clep,rly  indicates  no  effects 
that  can  be  attributed  to  their  training  with  the  American 
Management  Association. 

Prom  the  specific  LEA's  a  persistent  pattern 
emerges  once  again.    Personnel  in  ELI  express  a  higher 
level  of  participation  in  the  decisionmaking  proc-.ss 
within  the  district  than  is  expressed  in  the  other  LEA*s. 
This  difference  is  not  significant,  however.    Prom  a 
longitudinal  perspective,  every  LEA  in  this  study  showed 
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no  significant  change  in  their  decisionmaking  pattern. 
If  the  pattern  was  relatively  open  before  training,  it 
remained  so.    If  personnel  felt  that  they  were  not  appro- 
priately drav/n  into  the  affairs  of  the  school  district, 
their  attitudes  remained  so. 

As  we  have  indicated,  these  findings  should  be 
interpreted  with  every  caution  appropriate  to  a  research 
design  that  lacks  a  pre-test  with  a  Control  group. 


C.    Management  Team  Relations 

These  five  questionnaire  items  have  been  selected 
to  evaluate  management  team  relations: 

1.  My  group  works  hard  to  achieve  its  goals. 

Points  of  Time:    Tl,  T2,  T3,  T4  (The  other 
items  in  this  category  have  the 
same  four  points  of  *:ime.) 

2.  My  work  group  understands  what  we  are  trying 
to  achieve. 

3.  I  feel  my  group  works  v^ell  together. 

4.  I  really  feel  my  immediate  work  group  is 
getting  things  done. 

5.  When  differences  arise  in  my^work  group,  we 
have  good  ways  for  settling  them  ourselves. 
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Item  #1:     Fall  1970  to  Spring  1971 

Item  #1  attempts  to  measure  goal-directed  activity 
by  eliciting  reactions  to  the  statement  My  work  group 
works  hard  to  achieve  its  goals,    E xp e r imen ta 1  LEA ' s  # 3 
and  #4  apparently  worked  harder  than  LEA's  #1  and  #2. 
Scores  in  the  former  were  aggregately  higher  although  no 
change  occurred  on  a  pre /post-training  comparison.  The 
AMA  program  had  a  negative  impact  on  ELI  and  EL2,  how- 
ever.   Groups  in  those  organizations  put  out  signifi- 
cantly less  effort  after  training. 

Item  #1;     Spring  1971  to  Fall  1971 

Apparently  the  negative  result  of  AMA  training 
reported  in  T1-T2  in  ELl  and  EL2  had  been  reversed  by  the 
time  of  the  T2-T3  assessment.    Much  more  work  was  devoted 
to  educational  goals  in  Fall  1971  than  in  Spring  1971. 
In  fact,  the  output  reached  at  T3  was  even  greater  than 
either  Experimental  LEA  had  achieved  in  Tl.    This  consti- 
tutes a  positive  effect  of  learning  the  MiA  planning 
process      There  were  mixed  results  in  EL3  and  EL4;  one 
group  went  up  and  the  other  dropped.    Neither  change  was 
attributable  to  training. 

Item  #1;     Fall  1971  to  Spring  1972 

Our  assessment  at  this  time  can  be  more  definitive 
than  during  the  preceding  periods  due  to  the  addition  of 
Control  lea's.    As  before,  comparisons  that  use  these 
controls  engendered  the  same  conclusions  as  comparisons 
without  them.     Contrasting  ELl  with  EL2,  and  EL3  with 
EL4,  v;e  found  no  significant  differences  between  them  nor 
did  either  experience  any  changes  between  Fall  1971  and 
Spring  1972.    Based  on  this  evidence,  none  of  the  groups 
was  affected  by  AMA  training.    Adding  the  Control  LEA's 
confirms  this  judgment:     none  of  the  Experimental  LEA's 
changed  in  comparison  to  them.    Two  sets  of  data, provide 
the  same  results:     no  effects  could  reasonably  be  called 
the  responsibility  of  AMA. 

Item  #1;     Fall  1970  to  Spring  1972 

No  major  difference  existed  between  E1/E2  or  E3/E4 
in  Fall  1970,  before  training  began,  and  between  these 
same  organizations  in  Spring  1972,  after  it  ended.  ELl 
labored  hardest  to  achieve  their  goals,  follov/ed  by  EL3, 
EL4  and  EL2,  but  none  of  them  changed  over  time.  This 
absence  of  any  important  movement  one  way  or  the  other 
necessitates  a  firding  of  no  training  effects  on  this 
variable. 
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Item  #2:    Fall  1970  to  Spring  1971 

Organizational  comprehension  of  expected  results 
is  assessed  by  Item  ^2,  My  work  group  understands  vrhat  we 
are  trying  to  achieve >    Some  improvements  were  made-  in 
El3  after  training,  but  these  T^^cre  so  minimal  as  to  be 
unattributable  to  the  AMA  program.    Conversely,  EL4's 
grasp  of  intended  ends  contracted,  but  it  was  still  insuf- 
ficient for  assignment  of  any  effects.    Negative  effects 
occurred  in  ELl  and  EL2.    Both  Experimental  LEA's  declined 
significantly  between  Pall  1970  and  the  following  Spring; 
they  showed  less  understanding  of  expected  results  after 
training. 

Item  .^2:     Spring  1971  to  Fall  1971 

Little  alteration  was  noted  in  EL3  or  EL4  during 
the  post-training  period;  whatever  change  took  place  was 
insignificant,  aid  thus  unattributable  to  the  TiMA.  In 
T2-T3,  EL3  displayed  slightly  less  understanding,  while 
EL4  showed  somewhat  more.    However,  the  sizable  gains 
registered  by  ELl  and  EL2  in  1971  indicated  a  positive 
effect;  both  groups  notably  strengthened  their  grasp  of 
the  purpose  of  their  worko    Both  Experimental  LEA's  showed 
commensurate  gains. 

Item  #2;     Fall  1971  to  Spring  1972 

ELl,  EL2,  EL3,  and  E1.4  stabilized  their  levels  of 
understanding  between  Fall  1971  and  Spring  1972.  Although 
a  difference  of  some  significance  between  ELl  and  EL2 
was  revealed,  it  was  inadequate  justification  for  ascrib- 
ing any  effects  to  AMA  training.    EL3  and  EL4  also  did  not 
recast  their  comprehension  sufficiently  that  any  change  can 
be  attributed  to  training.    These  opinions  we}?c  strengthened 
by  continued  evaluation  of  Experimental  and  Control  LEA 
differences.    In  comparison  with  the  appropriate  Control 
group,  none  of  the  Experimental  LEA's  significantly 
differed,  nor  did  any  shift  over  time.    Therefore,  the 
direct  comparison  of  EL1/EL2  and  EL3/EL4  as  well  as  the 
comparisons  with  CLl  and  CL2  offered  the  same  conclusion: 
no  effect  of  training. 

Item  #2;    Fall  1970  to  Spring  1972 

A  review  of  felt  levels  of  understanding  before 
training  (Pall  1970)  and  of  the  same  levels  almost  a  year 
after  training  (Spring  1972)  provides  an  overall  assessment 
of  the  impact  of  AIIA  upon  this  variable.    While  the  LEA's 
that  entered  the  team  Planning  process  all  showed  limited 
growth  in  their  understanding  of  expected  results,  these 
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increase  wore  inconsequential.    For  this  reason,  the 
him  program  is  not  responsible  for  any  effects,  positive 
or  negative,  in  any  group. 

Item  #3?    Fall  1970  to  Spring  ir71 

Looking  at  Item  #3,  I  feel  my  groun  works  v;ell 
together,  during  the  immediate  pre-  and  post-training" 
periods  reveals  that,  with  one  exception,  all  LEA's  had 
less  satisfactory  team  relations  after  training.  EL3 
slightly  improved  its  work  group  relations.  However, 
only  one  such  change  was  significant  enough  to  attribute 
to         training.    ELI  and  EL2  showed  a  negative  training 
effect,  for  in  both  states    work  relations  deteriorated 
over  time;  EL2  fell  faster  than  ELI. 


Item  #3:    Spring  1971  to  Fall  1971 

Further  training  effects  appeared  during  this 
period.    ELI  and  EL2  reversed  the  negative  effect  noted 
above.    Their  work  team  relations  improved  significantly 
over  time,  rising  above  pre-training  levels.  Although 
EL3  and  EL4  also  improved  in  this  comparison,  the  change 
was  not  statistically  significant. 

Item  #3;    Fall  1971  to  Spring  1972 

ELI,  2,  3,  and  4  all  stabilized  over  this  period 
and  shov;cd  no  effects  attributable  to  AIIA  training.  The 
addition  of  the  Control  LEA's  here  demonstrates  no 
significant  differences  over  time  between  the  Experimental 
LEA's  and  the  Control  groups.    A  significant  difference 
between  EL3  and  Control  EL2  appears,  but  because  no 
significant  difference  is  shown  over  time,  no  training 
effect  can  be  attributed. 


Item  03;    Fall  1970  to  Spring  1972 

An  analysis  of  the  levels  of  favorable  work  rela- 
tions before  training  and  almost  a  year  afterward  reveals 
no  significant  changes  over  time  among  the  four  Experimen- 
tal LEA'S.    A  significant  difference  does  appear  between 
ELI  and  EL2  (holaing  time  constant),  but  no  significant 
change  appeared  over  time  and  thus  no  effects  of  AMA 
training  are  ascribed.    In  summary,  then,  the  LEA's  on 
this  item  remained  approximately  stable  throughout  our 
research  periods. 


375 


Item  Fall  1970  to  Spring  1971 

Item  #4^  I  really  feel  my  immediate  work  group  is 
getting  things  done t  attempts  to  measure  the  degree  to 
which  persons  feel  their  vjork  groups  accomplish  results* 
h  T1-T2  comparison  on  this  item  shows  a  significant 
negative  training  effect  between  ELI  and  EL2  over  time. 
These  two  school  districts  declined  in  a  statistically 
significant  way  in  the  degree  to  which  they  believed  they 
were  accomplishing  results.    FL3  and  EL4  remained  rela- 
tively stable  over  this  period,  showing  no  training  effects. 

Item  #4:     Spring  1971  to  Fall  1971 

A  resurgence  occurs  in  this  period  in  ELI  and  EL2. 
Over  time,  both  school  districts  attained  levels  higher 
than  their  pre-training  scores.    This  produced  a  positive 
training  effect.    ELS  and  EL4  continued  to  show  no  train- 
ing effects  in  this  comparison. 

Item  #4;    Fall  1971  to  Spring  1972 

No  significant  movement  occurs  over  this  time  in 
any  LEA.    Consequently  no  training  effects  can  be  ascribed. 
The  addition  of  the  Control  LEA's  shows  no  significant 
differences  between  the  LEA's,  either  holding  time  con- 
stant or  over  time.    In  general,  all  the  Experimental 
lea's  typically  stabilized  during  this  comparison. 

Item  #4;    Fall  1970  to  Spring  1972 

A  comparison  of  the  Experimental  LEA's  pre-training 
responses  with  those  a  year  after  training  indicates 
no  significant  effects  as  a  result  of  AMA  training.  The 
amount  of  work  people  felt  they  were  accomplishing  remained 
nearly  the  same:    All  LRA's  experienced  general  stabiliza- 
tion and  returned  to  their  pre-training  levols  by  Spring 
1972. 

Item  #5:    Fall  1970  to  Spring. 1971 

Asking  t'Then  differences  arise  in  my  work  group, 
we  have  good  ways  for  settling  them  ourselves #  Item  #5 
attempts  to  measure  the  degree  to  which  people  believe 
conflict  can  be  constructively  resolved  within  their  work 
groups.    A  T1-T2  comparison  shows  no  significant  training 
effect  on  this  item.    The  respondents  generally  saw  them- 
selves in  the  same  relative  position  on  this  issue  before 
and  immediately  after  training.    No  negative  or  positive 
training  effects  occurred. 
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Item  »5:    Spring  lg71  to  Fall  1971 

Between  ELI  and  EL2#  this  ccxnparison  shown  a 
significantly  positive  effect  of  RfVu  training  over  time. 
A  consistent  pattern  appears  in  both  LEA*s  in  a  T2-T3 
comparison.    The  ability  of  work  groups  to  handle  conflict 
improved  significantly  between  Spring  and  Fall  1971. 
ELS  and  EL4  also  follow  a  pattern  that  is  linked  to  their 
general  management  team  relations #  and  they  show  no  change 
over  time  or  between  thomsolves. 

Item  #5:    Fall  1971  to  Spring  1972 

It  appears  that  the  rapid  improvement  in  conflict 
management  in  ELI  and  EL2  reverses  itself  in  a  T3-T4 
comparison.    When  these  LEA*s  are  compared  v;ith  the  Con- 
trol group ^  however^  a  positive  effect  of  AMA  training 
emerges.    While  both  Experimental  LEA's  are  dropping,  the 
Control  LEA  is  falling  at  a  faster  rate.    As  we  have 
indicated  before,  this  data  must  be  interpreted .cautiously 
because  of  the  lack  of  pra^-training  data  for  the  Control 
LEA.    But  EL3  and  EL4  again  demonstrate  no  effect  of 
training  when  compared  x^ith  each  other  or  with  the  Con- 
trol LEA. 


Item  #5;    Fall  1970  to  Spring  1972 

A  comparison  of  pre-training  data  with  data  recorded 
in  T4  indicates  no  effects  of  AtlA  training  on  this  item. 
Groups  in  these  school  districts  were  apparently  no 
better  able  to  handle  conflict  in  their  work  groups  after 
training  than  before.    No  significant  changes  were  recorded 
either  between  the  LEAs  or  over  time. 
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DATA  SU;4IlARy 
Management  Team  Relations 
Fall/  1970  -  Spring^  1972 

Type  of  Data 
Item  QOESTIOIWAIRE 


iriPACT  OP  TRAINING 


Positive 
Effect 


1  My  work  group  works  hard 
to  achieve  its  goals, 

2  My  work  group  under- 
stands what  we  are  try- 
ing to  achieve, 

3  I  feel  my  group  works 
well  together, 

4  I  really  feel  my  imme- 
diate v/ork  group  is 
getting  things  done, 

5  When  differences  arise 
in  my  vjork  group  /  we 
have  good  w;=>ys  for 
settling  them  ourselves. 


NO 

Effect 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3^EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


Negative 
Effect 


These  five  items  were  designed  to  measure  the 
extent  to  which  the  7JlA's  process  goals  for  the  develop- 
ment of  a  management  team  were  accomplished.    The  longitu- 
dinal perspective  of  this  study  clearly  indicates  that 
AMA  training  did  not  influence  the  team  relations  in 
these  school  districts  either  positively  or  negatively. 

Items  #1  and  #2,  which  rf^fer  specifically  to  the 
clarity  of  goals  within  work  groups,  reveals  that  ELl's 
groups  understood  most  clearly  their  aims.    All  the  LEA's 
experienced  limited  growth  in  their  knowledge  of  expected 
results,  but  in  no  case  was  this  significant  enough  to 
attribute  to  AMA  training. 

Items  #3,  #4,  and  S5  Wv'iro  designed  to  reflect  the 
attitudes  of  group  members  regarding  their  ability  to  work 
together  and  to  resolve  emergent  conflicts.    Again  we 
find  a  pattern  of  general  stabilization  over  tii\iO;  all 
the  LEAS  have  reverted  to  their  approximate  nre-training 
levels  by  the  Spring  of  1972, 
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Our  ovorall  conclusion  regarding  this  category 
of  variables  is  that  riik  training  simply  has  no  effect 
on  modes  of  personal  relations  within  work  groups  in 
the  organizations  that  are  trainedl    in  many  cases, 
LEA'S  showed  low  mean  scores  in  the  immediate  post- 
training  period,  Afhich  over  the  course  of  the  second 
year  of  this  evaluation  regained  their  pre-training 
levels. 


Section  4:    Summary  and  Conclusions 

Our  purpose  in  Chapter  Five  was  to  evaluate  the 
extent  to  which  the  WlA  training  process  affected  the 
internal  health  of  the  Experimental  organizations.  The 
AMA  has  specific  procef^s  goals  for  leadership  climate, 
decisionmaking,  and  manaorement  team  relations  which  we 
wished  to  evaluate. 

An  f-^ttempt  is  made  in  the  training  design  to 
instill  modern  concepts  of  management,  including  the 
developmont  of  strong  patterns  of  leadership  which  sup- 
port the  organization's  goals;  decentralization  of 
decisionmaking  so  that  people  who  are  responsible  for 
decisions  actually  make  them;  and  the  development  of 
teams  of  managers  who  cem  work  together  constructively 
to  achieve  the  goals  of  the  organization  (which  should 
be  the  goals  of  the  team) . 

The  importance  of  these  intervening  variables  was 
established  earlier.    Our  purpose  here  is  not  to  provide 
further  evidence  of  their  central  place  in  the  transforma 
tion  of  training  input  into  planning  output.    We  intend 
now  to  determine  to  what  extent  AMA  has  been  able  to 
affect  positively  these  variables.    Has  training  bred  an 
organizational  environment  into  which  the  team  planning 
process  can  successfully  be  introduced? 
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SUMMARY  OP  ATTITUDINAL  FINDINGS 


INTERVENING  VARIABLES  -  STATE 
DEPARTMENT  OP  EDUCATION 

Fall,  1970  -  Spring,  1972 


IMPACT  OP  TRAINING 


Positive  No 
Effect  Effect 


Item  Type  of  Data 

LEADERSHIP  CLIMATE 
QUESTIONNAIRE 

1  Based  on  information  I  have 
received  from  my  boss,  I 
know  if  I  am  measuring  up 

in  my  job#  El 

2  My  manager  encourages  euid 
supports  innovation.  El 

3  Higher  management's  reac- 
tions to  the  problems  which 

reach  them  are  fair*  E2  El 

4  My  manager  knows  and  under- 
stands the  problems  I  face.  El 

5  My  manager  recognizes  when 
a  problem  is  developing  and 
does  something  constructive 
about  it.  El 

6  My  manager  shows  confidence 

and  trust  in  me.  El 

DECISIONIiAKING 
CONTENT 

1  Involvement  in  Decision- 
making in  the  State 

Department  E1,E2 

2  Quality  of  Decisionmaking 

in  the  State  Department  E1,E2 

3  Influence  of  Planning  on 
decisiopmaklng  process  E1,E2 

QUESTIONNAIRE 

4  The  people  I  work  with 
participate  appropriately 
in  setting  the  goals  of 

our  work.  El 

5  I  am  appropriately  involved 
in  decisions  affecting  my 

work.  E1,E2 


Negative 
Effect 


E2 
E2 


P2 

E2 
E2 


E2 
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SUMMARY  OF  ATTI'JUDINAL  FINDINGS  (cont'd) 

IMPACT  OF  TRAINING 


Item 


1 
2 

3 
4 

5 
6 


Type  of  Data 

I  can  influence  the  goals, 
methods,  and  activities 
of  my  organization. 

MANAGEflENT  TEATl  RELATI(»IS 

CONTENT 

Promote  cooperative  team- 
work 

Amount  of  cooperative 
teamwork 

QUESTIONNAIRE 

My  work  group  works  hard 
to  achieve  its  goals. 

My  work  group  understands 
what  we  are  trying  to 
achieve. 

I  feel  my  group  works  well 
together. 

I  really  feel  my  immediate 
work  group  is  getting 
things  done. 

When  differences  arise  in 
my  work  group,  we  have 
good  ways  for  settling 
them  ourselves. 


Positive 
Effect 


No 
Effect 


Negative 
Effect 


El 


E1,E2 


E2 


El 


E2 


E1,E2 


El 
E2 


E2 


El 
El 


E2 


E2 


El 


Mixed  effects  of  training  on  the  SED  intervening 
variables  appear.    Except  for  items  associated  with  lead- 
ership climate,  no  clear  trends  are  apparent  when  one 
scans  the  data  summary  charts.    But  subsequent  examina- 
tion reveals  two  definite  patterns,  each  connected  with 
an  Experimental  State. 

One  pattern  which  becomes  evident  is  the  positive 
effects  attributable  to  training  in  El.    Items  related  to 
the  three  categories  reveal  that,  while  the  AMA  did  not 
affect  many  of  them,  it  had  positive  effects  on  some  and 
negative  effects  on  none.    After  training,  leadership 
climate  improved  most.    Persons  in  that  organization  felt 
better  about  their  management  team's  leadership  style 
after  training,  and  this  improvement  was  sufficient  to  be 

ERIC 
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defined  as  a  positive  training  effect  in  five  of  the  six 
itoms  associated  with  this  category. 

The  other  pattern  which  emerges  is  the  continual 
negative  impact  of  training  on  F2'8  intervening  variables. 
Items  associated  with  leader shin  climate,  decisionmaking 
and  management  team  xelations  were  adversely  affected  by 
training  in  E2.    This  negative  consistency  does  not  augur 
well  for  planning  (which  will  bo  evaluated  as  an  end 
result  variable  in  Chapter  Six) .    What  may  be  harmed  are 
future  plans  to  be,  or  not  to  be,  developed,  as  well  as 
the  potential  support  of  people  in  the  organization — a 
prerequisite  of  transforming  paper  plans  into  action  plans. 


SUMMARY  OF  ATTITUDINAL  FINDINGS 


DATA  SUMMARY 

INTERVENING  VARIABLES  -  LOCAL 
EDUCATION  AGENCIES 

Fall,  1970  -  Spring,  1972 


Item  Type  of  Data 

QUESTIONNAIRE 
LEADERSHIP  CLIMATE 

1  Based  on  information  I  have 
received  from  my  boss,  I 
know  if  I  am  measuring  up 
in  my  job. 

2  My  manager  encourages  and 
supports  innovation. 

3  Higher  management's  reactions 
to  the  problems  which  reach 
them  are  fair. 

4  My  manager  knows  and  under- 
stands the  problems  I  face. 

5  My  manager  recognizes  when  a 
problem  is  developing  and 
does  something  constructive 
about  it. 

6  My  manager  shows  confidence 
and  trust  in  me. 


IMPACT  OF  TRTilNING 

Positive       No  Necative 
Effect      Effect  Effect 


rLl,EL2 
EL3,EL4 

EL1,EL2, 
DL3,EL4 

rLl,EL2, 
FL3,T?L4 

EL1,EL2, 
EL3,EL4 


EL1,EL2  EL3,EL4 

EL1,EL2, 
EL3,EL4 


DECISIONMAKING 

The  people  I  work  with  parti- 
cipate  appropriately  in  set- 
ting the  goals  of  our  work. ' 


EL1,EL2, 
EL3 ,EL4 
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SUMIi/iRY  OF  ATTITUDINriL  FINDINGS  (cont'd) 

IMPACT  OF  TRAINING 


Positive       No  ^^egative 
Item  Type  of  Data  Effect     Effect  Effect 

QUESTIOITOAIRE 

2  I  am  appropriately  involved 

in  decisions  affecting  my  EL1,EL2, 

work.  ELSyEL'l 

3  I  can  influence  the  goals, 

methods,  and  activities  of  EL1,EL2, 

my  organization.  EL3,EL4 

IIANAGEMENT  TEAM  RELATIONS 

1  My  work  group  v7orks  hard  to  EL1,EL2, 
achieve  its  goals.  .EL3,EL4 

2  My  work  group  understands  what  EL1,EL2, 
we  are  trying  to  achieve.  EL3,EL4 

3  I  feel  my  group  works  well  EL1,EL2, 
together.  EL3 ,EL4 

4  I  really  feel  my  immediate 

work  group  is  getting  EL1,EL2, 

things  done.  EL3,EL4 

5  When  differences  arise  in 
my  work  group  we  have  good 

ways  for  settling  them  EL1,EL2, 

ourselves .  EL3 , EL4 


LEA 

Turning  to  the  Local  Educational  Agencies  (LEA*s) 
in  our  analysis  of  the  effects  of  AMA  training  on  the 
intervening  variables,  we  are  faced  with  striking 
results.    With  only  two  exceptions,  no  statistical  evi- 
dence of  AMA  training  effect  appeareT^in  the  Experimen- 
tal LEA*s;  the  two  effects  recorded  were  negative.  In 
other  words,  the  two  LEA*s  in  question  were  worse  off 
after  training  on  these  items  than  they  had  been  before 
training. 


Our  analysis  makes  it  quite  clear  that  the  AI-IA 
training  program  simply  does  not  have  any  significant 
effect  over  time  on  these  variables.    This  is  not  to  say 
that  the  Experimental  organizations  do  not  practice  these 
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managomGnt  techniques;  in  fact,  at  least  one  LEA  evinced 
a  consistent  pattern  of  very  high  mean  scores  on  most 
items.    Our  results  do  say  that,  if  an  organization 
practices  competent  management  techniques  before  Mili 
training,  it  will  continue  to  practice  them  afterward, 
showing  no  appreciable  effect  from  the  AIIA  process.  On 
the  other  hand,  an  orgcinization  with  a  low  level  of 
managerial  skills  will  not  benefit  from  AMA  training, 
either.    It  may  well  continue  to  preserve  dysfunctional 
management  patterns. 

In  closing,  we  must  point  out  again  that,  while 
we  have  confidence  in  our  findings,  we  caution  readers  to 
remember  that  the  research  design  upon  which  our  conclu- 
sions are  based  is  not  as  pot*7erful  as  we  would  like.  As 
discussed  in  Chapter  Two,  a  research  design  that  lacks 
a  pre- training  Control  Group  test  has  limitations.  How- 
ever, the  addition  of  the  Control  LEA's  in  the  T3-T^ 
.comparison  gives  us  confidence  in  these  conclusions  that 
nearly  equals  our  faith  in  the  conclusions  regarding  the 
SED  data. 


Overall  Conclusion 

The  effects  of  training  on  the  Educational  Agen- 
cies' intervening  varieiblcs  arc  diverse  enough  to  defy  absolute 
interpretation.    Positive,  negative,  and  no  effects  are 
spread  across  the  items  and  organizations  without  any 
dominant  pattern* 

No  brief  statement  could  accurately  describe  what 
we  observed  concerning  the  intervening  variables,  and  a 
longer  explanation  would  violate  the  purpose  of  a  summary. 
It  can  be  argued  that  any  movement  in  this  complicated 
area  indicates  a  successful  A.'^iA  intervention.    But  if  we 
assume  (as  we  must)  that  AI^iA  wanted  to  create  positive 
training  effects  upon  these  organizations*  intervening 
variables,  then  the  results  and  conclusions  are  indeed 
mixed.    For  the  AllA  partially  succeeded,  they  partially 
failed,  and  in  part  they  showed  no  effect  at  all. 

Training  influenced  the  "internal  state  and  health" 
of  each  organization  in  different  v;ays.    Yet  if  we 
categori5:e,  and  perhaps  oversimplify,  the  data  by  organi- 
zation, then  the  terms  "success,"  "failure"  and  "no  effect" 
assume  more  meaning.    It  then  becomes  evident  that  train- 
ing influenced  the  intervening  variables  of  El  positively, 
E2  negatively,  and  of  the  four  LEA's  hardly  at  all.  The 
AMA  produced  one  set  of  successful  results  out  of  six 
possible  opportunities. 
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CHAPTER  SIX 


ENE)  RESULT  VARIABLES 


Overview 

The  central  concern  of  this  evaluation  is  the 
inipact  of  the  AI4A  training  program*    We  have  defined 
" impact in  two  ways*    In  the  chapter  on  causal  variables # 
we  viewed  it  as  satisfaction  of  the  AMA  goal  criteria, 
ranging  from  defining  the  institution's  mission  to  design- 
ing em  evaluation  methodology.    " Impact became  improve- 
ments in  the  internal  decisionmaking  process  and  orgami- 
zatipnal  climate  in  the  intervening  variables  chapter. 

Now  we  define  "impact"  in  still  another,  and  perhaps 
most  important,  way.    This  chapter  on  end  result  variables 
ex2uaines  impact  as  the  "achievements  of  the  organization,  ^ 
such  as  services  provided  or  goods  produced.    Given  change 
or  lack  of  change  in  the  causal  and  intervening  orgemiza- 
tional  variables,  how  has  AMA  training  affected  what  the 
educational  agencies  actually  produce  for  their  own  use 
or  their  clients'  benefit? 

Improvements  in  how  an  institution  makes  strategic 
action  assignments  or  deals  with  interpersonal  conflict 
are  essential  to  organizational  planning  and  growth,  but 
such  improvements  can  be  considered  as  a  means  to  another 
end:    the  design  and  implementation  of  a  long-range 
strategic  plan. 2    Unless  plans  are  being  written, 
other    efforts     to  build  an  effective  educational  plan- 
ning process  become  virtually  meaningless. 


■^Rensis  Likert,  The  Human  Organizati  n  (New  York: 
McGraw-Hill,  1967),  p.  29. 

^American  Management  Association,  "Feasibility  and 
Pilot  Programs  Proposal:    A<1apting  and  Testing  Business 
Management  Development  Programs  for  Educational  Adminis- 
trators" (mimeograph),  June  22,  19l0,  pp*  ^^^5.    The  14th 
criterion  on  which  the  AMA  program  was  to  be  evaluated 
was  the  extent  to  which  t:he  experimental  agencies  had 
"•produ^ted  and  are  implementing  a  long-range  strategic 
plan." 
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Furthermore,  we  should  study  the  extent  to  which 
administrative  objectives  specified  in  the  plan  are  be- 
ing achieved.    VJhile  plans  constitute  a  legitimate  output 
of  the  training,  they  are  clearly  a  means  to  the  end  of 
being  able  to  measure  progress  toward  the  agencv's 
administrative  objectives.    To  be  sure^  vie      -     .  exist 
before  progress  can  be  made  toward  objecti  ^ac 
measurable  progress  is  a  better  test  of  MiTi  training 
effectiveness  than  plans  alone  are.    Asserting  that  the 
ESED  will  "By  September  1971  assist  in  the  establishment 
of  35  new  Kind irgarten-Early  Childhopd  Education  Centers" 
is  one  thing;  ^eing  able  to  show  th^t  this  objective 
has  been  accomt"  lished  shows  Mik  training  impact  of  a 
higher  order.    Without  measurable  progress,  plans  may  be 
worthless  paper.    Both  plans  and  progress  are  appropriate 
end-result  variables  but  one  is  a  more  substantial  end 
result  than  the  other. 

Even  plans  and  progress  tc^'^ard  administrative 
objectives  are  not  the  last  links  in  this  extended  means-- 
ends  chain.    An  ultimate  test  of  the  worth  of  the  ?J12l 
rrogram  would  be  "the  output  of  the  system  measured  in 
terms  of  the  skills  and  aptitudes  transmitted  to 
students  in  the  educational  systems. "3    But  while  many 
objectives  are  written  for  student  outcomes,  evaluating 
whether  they  are  being  achieved  is  premature  at  this  time. 

We  can  look  at  progress  toward  some  student  objec- 
tives more  than  a  year  after  training  ended  (especially 
on  the  strategy  level) .  Yet  a  more  comprehensive  judg- 
ment of  effects  of  the  AMA  team  planning  process  on  the 
education  of  students  must  wait  until  the  agencies  have 
had  ample  time  to  implement  the  objectives  enumerated  in 
their  action  plan.  Rensis  Likert  identified  this  problem 
in  his  own  stud4.es: 

.  .   .  the  time  intervals  required 
before  both  productivity  began  to 
increase  and  before  the  situation 
reached  a  state  of  relative  equi- 
librium was  appreciably  longer  than 
those  conducting  the  work  had  ex- 
pected.   Changes  in  the  causal 
variables  .  .  .  apparently  require 
an  appreciable  period  of  time  before 


Jesse  Burkhead  (with  Thomas  G.  Fox  and  John  W. 
Holland) ,  Input  and  Output  in  Large  City  High  Schools 
(Syracuse:    Syracuse  University  Press,  1967).    See  also: 
Raymond  E.  Klawuhn  and  Alexander  J.  Basso,  "Final  Report: 
Adapting  and  Testing  Business  Management  Development  Pro- 
grams for  Educational  Administrators"  (mimeograph) ,  Jan. , 
1972,  p.  30. 
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^rhe  impact  of  the  change  is  fully 
n»e.ni*lest  in  corresponding  improveinent 
in  the  end  result  variables,^ 

Time  becomes  an  even  more  important  factor  when  the  objec*- 
tives  are  not  scheduled  for  completion  until  after  our 
evaluation  is  published;  most  of  the  student  behavioral 
objectives  in  these  plans  are  not  expected  to  be  reached 
until  after  we  go  to  press.    A  report  written  in  mid-1972 
cannot  too  precisely  assess  the  extent  to  which  ESED  #2 
has  accomplished  this  goal:    "By  1977,  85%  of  all  15  year 
old  students  will  demonstrate  cOTiputational  skills  as 
determined  by  appropriate  criterion  referenced  tests." 
Our  concern  will  be,  therefore,  to  measure  progress 
toward  goals. 

This  is  not  to  suggest  a  need  for  a  Y3  evaluation 
program.    Making  strategic  action  assignments  is  an 
essential  precondition  for  the  design  of  long-range  plans. 
Likewise,  an  assessment  of  plan  quality  and  preliminary 
indications  of  early  progress  toward  its  administrative 
realization  provide  a  clue  to  future  developments  and 
directions. 

.  .  ,  the  available  evidence  indicates 
that  there  are  consistent  and  dependable 
relationships  among  the  causal,  interven- 
ing, and  end  result  variables.    When  all 
of  the  relevant  factors  are  taken  into 
consideration,  especially  time,  and  the 
proper  analyses  made,  consistent  positive 
relationships  can  be  expected  among  the 
causal,  intervening,  and  end  resylt 
variables  in  every  organization. ^ 

Therefore,  this  chapter  (1)  compares  selected  plans 
with  established  criteria  for  good  planning  and  (2)  reviews 
progress  toward  those  objectives  and  strategies  with 
approximate  completion  dates. 

Chapter  Six  is  divided  into  five  (5)  sections.  The 
first  three  (3)  concentrate  on  the  ESED's.    Section  1 
evaluates  the  plans  themselves  while  Section  2  measures 
progress  toward  objectives.    For  purposes  of  logical 
presentation,  we  title  Sedtion  1    Intermediate  Output"  and 
Section  2  "Final  Output." 

Section  3  revjuews.the  extent  to  which  persons  within 
the  ESED's  feel  they  have  produced  a  long-rango  strategic 


^  Likert,  0£.  ci^t. ,  pp.  80-81. 

^Ibid.,  pp.  98-99. 
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plan,  as  well  as  what  progress  they  sense  has  been  made 
toward  the  objectives  of  the  plan. 

Section  4  focuses  on  the  Experimental  LEA's.  Here 
we  evaluate  the  plans  produced  as  a  result  of  AM?1  training, 
against  the  same  criteria  used  for  the  ESED's.    We  also  look 
at  what  participants  think  about  the  plans,  and  offer 
conclusiv')ns  on  the  impact  of  AMA  training  on  LEA  end-result 
variables . 

Section  5  integrates  conclusions  from  previous 
sections.    It  presents  the  overall  conclusions  of  the 
research  team  and  a  comprehensive  sunmary  of  our  findings 
and  analysis  in  this  chapter. 


Section  1;    Intermediate  Output/SED 

To  evaluate  the  plana  produced  by  two  Experimental 
State  education  agencies  is  no  mean  task.    Each  agency 
has  numerous  bureaus,  sections,  divisions,  and  offices. 
Attempting  to  study  them  all  is  beyond  the  resources  of 
this  research  team.    It  would  also  be  irrelevant  to  the 
actual  status  of  the  planning  process  in  the  Experimental 
agencies,  since  many  units  have  not  yet  produced  or  com- 
pleted action  on  their  plans. 

In  order  to  obtain  a  representative  sample  of 
plans  that  would  be  fair  to  the  agencies  as  well  as  con- 
sistent with  the  purposes  of  this  evaluation,  we  chose  to 
concentrate  on  the  priority  areas  established  by  the 
ESED's.o    Those  were  most  likely  to  have  been  fully 
developed.    Priority  areas  are  usually  accorded  greater 
attention  and  more  resources.    If  these  plans  were  unsatis- 
factory, we  could  reasonably  assume  that  goals  with  lower 
priority  would  be  similarly  weak.    On  this  basis,  the 
plans  chosen  for  evaluation  were  those  written  for: ' 


Priorities  for  ESED  #1  are  largely  informal,  while 
those  for  ESED  #2  are  formal  and  have  been  widely  publicized 
throughout  the  educational  system. 
7 

Each  state  was  asked  to  provide  the  research  team 
with  current  plans  for  their  tsriority  areas.  Plans  were 
not  received  for  ESED  #1*8  Early  Childhood  Education  pro- 
gram, so  it  cannot  be  included  at  this  stage  of  analysis. 
Fortunately,  its  division  director  made  a  special  report 
on  progress  toward  objectives  and  strategies  specified  by 
the  plan  with  proximate  completion  dates.  This  data  will 
be  used  in  the  Final  Output  section. 


ESEO  #1 


ESED  #2 


Career  Education 
Early  Childhood 
Reading 


Early  childhood 
Human  Relations 
Reading 
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Drawing  on  several  sources,  the  research  team 
developed  criteria  by  which  to  assess  the  fonnal  quality 
of  these  plans. 8    They  encompass  the  essential  elements 
of  planning  documents;  each  criterion  bears  upon  the  subse- 
quent acceptance  and  success  of  the  plan. 


CRITERION 


INDICATOR  OP 


1*  Title; 


2 •    Identification : 


3.    Needs  Assessment: 


serves  to  identify  the  program  and 
its  sponsoring/operating  unit. 

persons  preparing  and  approving  plan. 
The  person (s)  assigned  responsibil- 
ity  for  implementing  a  plan  will 
want  to  know  the  names  of  the  per- 
sons who  prepared  the  plan  in  case 
he  needs  their  assistance.    It  is 
also  useful  to  know  by  whose  author- 
ity the  plan  has  been  approved.  This 
will  give  implementers  some  indica- 
tion of  the  importance  of  the  plan 
as  well  as  give  it  organizational 
legitimacy. 

short  statement  of  the  unmet  needs 
for  which  the  plan  is  designed.  A 
need  is  the  difference  between  cur- 
rent conditions  and  condition?  that 
are  desired.    A  clear  needs  assess- 
ment provides  the  rationale  for  the 
plan's  existence  and  states  its 
contribution  to  meeting  the  need. 


ERIC 


g 

Primary  source  was  Preston  P.  LeBreton  and  Dale  A. 
Henning,  Planning  Theory  (Englewood  Cliffs,  N.J.:  Pren- 
tice-Hall,  Inc*,  1961). 

Secondary  sources  included  Robert  N.  Anthony, 
Planning  and  Control  Systems;    A  Framework  for  Analysis 
(Boston:    Harvard  University  Press,  1065);  Bertram  M. 
Gross,  Organizations  and  their  Managing  (New  York:  The 
Free  Press,  1968);  Daniel  Stuff lebeam  et  al..  Educational 
Evaluation  and  Decision  Making  (Bloomington,  Indiana: 
Phi  Delta  Kappa,  Inc.,  1971);  Joseph  S.  Wholey  et  al..  Fed- 
eral Evaluation  Policy  (Washington,  D^C:    The  Urban  Insti- 
tute, 1970). 

Organizational  Documents  included  Joe  Wolvek, 
Comprehensive  Planning  in  State  Education  Agencies  (Des 
Moines,  Iowa:    Iowa  Department  of  Piiblic  Instruction, 
1968);  ''Educational  Nc^ds  Assessment:    A  Statewide  Design 
for  Texas"  (Texas  Education  Agency,  1971) ;  George  Muench 
et  al.,  '•Educational  Goals  and  Objectives"  (California 
School  Boards  Association,  1969). 
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CRITERION 
Recommendations : 


Expectod  Results: 


Time  Frames: 


Respons  ibi 1 i ty : 


8.  Resource 

Requirements: 


ERiC 


INDICATOR  OF 

enumeration  of  what  ought  to  be  done 
to  meet  identified  needs.  Recom- 
mendations may  be  short-  or  long- 
range,  timeless  or  continuous  in 
nature.    Recommendations  should  be 
organized  into  a  means-ends  chain 
linking  broad  statements  of  purpose 
(mission) ,  intermediate  goals  (con- 
tinuing and  specific  objectives) 
and  specific  methods  for  achieving 
them  (strategies) .    Closely  con- 
nected with,  if  not  included  in, 
recommendations  are  the  following 
plan  elements: 

anticipated  payoffs  from  the  adoption 
cf  any  recommendations  expressed 
in  roeasureible  terms,  specifying  who 
will  benefit  (clients)  and  by  how 
much.    This  serves  as  a  standard  by 
which  to  evaluate  success  of  the 
plan.    It  also  justifies  the  re- 
sources requested  to  implement  it. 

an  indication  of  when  recommendations 
are  to  be  implemented  and  when  they 
are  expected  to  be  complete.  While 
less  important  than  the  completion 
date,  the  implementation  date  indi- 
cates the  time  available  to  manage- 
ment to  prepare  chems elves  before 
action  on  the  plan.    The  completion 
date  serves  as  another  basis  on  v;hich 
to  evaluate  the  plan;  a  comparison 
of  actual  and  expected  cc»npletion 
dates  is  an  indicator  of  plan  quality* 

assignment  of  each  recommendation 
to  a  specific  person(s)  group(s), 
or  operating  unit(s).  Objectives 
assigned  to  everyone  frequently  becane 
the  responsibility  of  no  one. 

an  estimate  of  the  personnel,  money, 
material,  or  information  needed  to 
attain  each  recommendation.  Provides 
superiors  with  some  indication  of 
how  much  the  plan  might  cost,  as  well 
giving  a  basis  on  which  to  compare: 

(a)  expected  results  with  costs 

(cost/benefit  study) 

(b)  alternative  uses  of  the  same 

resources. 
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CRITERION 


INDICATOR  OF 


9.  Evaluation 
Strategics: 


10.  Rationale: 


methods  used  to  measure  progress 
toward  recommendations.    An  attached 
evaluation  strategy  gives  persons 
responsible  for  plans  (and  those  to 
v;hom  they  are  responsible)  reliable 
data  on  the  extent  to  vrhich  recom- 
mendations are  attained.    Lack  of 
such  strategies  makes  it  difficult 
to  separate  programs  that  work  and 
should  be  funded  from  those  that  do 
not  and  must  be  eliminated. 

justification  ^or  elements  in  the 
plan.    The  purpose  is  to  provide 
answers  to  possible  questions  about 
the  plan.    Rationale  can  focus  on 
what  recommendations  will  accomplish, 
justify  a  plan's  cost,  or  explain 
the  time  frames  or  responsibility 
center.    Any  supporting  data  or  nar- 
rative that  could  clarify  doubts  on 
these  issues  should  be  presented. 


Analysis 

Each  available  plan  is  compared  with  the  above 
criteria.    To  simplify  analysis,  each  plan  is  rated  on  a 
"yes"  or  "no"  basis  as  to  whether  or  not  it  meets  each 
criterion.    A  "yes"  does  not  necessarily  mean  complete 
success,  nor  does  a  "no"  indicate  total  failure.    They  do 
indicate  whether  most  of  the  plan  fares  well  or  poorly 
when  held  against  the  corresponding  criterion.  Therefore, 
plans  should  be  compared  only  with  the  criteria,  not  with 
each  other. 

After  the  following  schematic  presentation,  tha 
analysis  is  briefly  explained. 
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/ANALYSIS  OF  ESED  PRIORITY  AREA  PLANS 
Spring,  1972 


ESED  #1 


ESED  #2 


Career 
Educa- 
tion 


Read- 
ing 


Early 
Child- 
hood 


Human 
Rela- 
tions 


CRITERIA 


A     A  L  V  5TT 


Read- 
ing 


1.  Title  Yes 

2.  Identifi- 
cation No 

3.  Needs  As- 
sessment Yes 

4 .  Recommen- 
dations No 

5 .  Expected 
Results  Yes 

6 .  Time 

Frames  Yes 

7 .  Responsi- 
bility  No 

8.  Resource 
Require- 
ments No 

9 .  Evalua- 
tion 

Strategies  No 
10.    Rationale  No 


Yes 

No 

Yes 
Yes 
Yes 

Yes 
No 

No 

Yes 

No 


Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 
No 


Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 
No 


Yes 

Yos 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

No 
NO 


Yes=s4        Ye8»6  Yes=5        Yes=5  yes=7 

No  =6       No  »4  No  »5       No  »5       No  =3 
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Coimnonts 

Both  ESED's  produced  moderately  acceptable  plans. 
Both  sets  of    recommendations  were  written  according  to 
AMA  specif ications ,  exnressing  missions/continuing  objec- 
tives/specific objectives  in  behavioral  terms. ^  Ml 
plans  include  expected  results  and  time  frames^  in  vary- 
ing degrees. 10 

By  breaking  out  its  objectives  and  strategies  in 
separate  sections  and  clearly  displaying  time  frames ^ 
ESED  #l«s  plan  is  superficially  clearer.    But  eSED  #2  has 
the  advantage  of  giving  more  detail  about  why  the  plans 
are  needed,  as  well  as  other  general  explanations. 

Neither  state  was  Vf-ry  specific  about  resources 
required  to  achieve  each  objective/strat'^gy,  though  the 
costs  of  recommended  prcgra^'^s  were  somewhat  considered. ^ 
The  research  team  has  seen  budcet  documents  from  both 
states,  but  these  are  separate  from  the  state's  plan.  It 
would  have  been  advantageous  to  attach  a  price  tag  and 
manpower  requirements  to  each  objective/strategy  in  the 
nlan,  rather  than  to  isolate  costs  from  recommendations 
in  separate  documents. 


The  plan  for  ESED  #l's  Career  (Occupational) 
Education  division  contains  two  (2)  sets  of  continuing 
objectives;  one  is  timeless  while  the  other  sets 
specific  dates.    No  specific  objectives  are  listed. 

^^As  mentioned  previously,  ESED  #2*s  plan  follows 
the  liMk  format  for  internal  purposes  only.    For  communi- 
cating with  external  constituencies,  particularly  the 
State  Board,  it  has  largely  abandoned  the  AIIA  breakdown 
of  mission/continuing  objectives/specific  objectives, 
etc.  in  favor  of  narrative  descriptions  of  programs  and 
non-behavioral  goals  and  objectives.    To  keep  plans 
consistent,  the  ESED  #2  internal  plan  was  used  for 
comparison  with  the  criteria;  as  was  ESED  #l*s  plan. 

^^In  ESED  #1,  each  plan  has  a  heading  "Required 
Personnel,  Equipment,  Etc."    But  under  each  heading, 
generalized  action  assignments  like  "staff"  or  "selected 
project  personnel"  appear  rather  than  hard  data  on 
costs.    Its  career  education  plan  makes  a  limited  at- 
tempt to  specify  amount  of  "required  staff  time"  beside 
objectives. 
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The  evaluation  strategies  suggested  were  also 
weak,    often  the  evaluation  strategy  listed  was  ex- 
pressed in  vague,  general  terms  like  "annropriate 
criterion-referenced  test."    Elsev;here  the  objective/ 
strategy  consisted  of  persons  completing  a  program, 
receiving  money,  or  meeting  other  task  completion 
inventories,  so  no  elaborate  evaluation  strategy  was 
inevitable.    A  few  evaluation  strategies,  although 
listed  in  the  planning  document,  were  not  attached  to 
the  objectives/strategies  they  might  have  tested.  Both 
states  included  future  production  of  appropriate  measures 
among  their  other  objectives.    The  situation  may  thus 
improve  once  evaluation  strategies  are  developed. 

Proper  identifications  were  rar-sly  made.    A  few 
plans  implied  who  might  have  prepared  or  approved  the 
plan,  yet  seldom  was  this  explicit.       Neither  state 
detailed  the  rationale  for  choosing  one  objective  over 
another . 

Needs  Assessments  vfere  often  in  prose  form  accompan- 
ied by  modest  empirical  evidence.     In  some  plans,  how- 
ever, it  was  difficult  precisely  to  delineate  the  need 
because  current  conditions  were  not  stated.    For  example, 
ons  ESED  might  have  said  that  85%  of  its  students  should 
be  able  to  use  reading  as  a  communications  skill.  The 
agency  did  not  say  what  percent  can  read  now.  This 
omission  gives  no  idea  of  what  percentage  gap  must  be 
closed  between  the  present  and  future  expectations. 


Section  2;     Final  Output 

At  this  point  in  time,  after  the  conclusion  of  aM7i 
training,  final  output  has  been  appropriately  defined 
as  measured  progress  toward  selected  objectives/strate- 
gies that  are  specified  in  the  plans.    Attitudinal  data 
reveal  how  much  progress  participants  feel  thoy  have 
made  toward  achieving  their  own  objectives/strategies.  - 

The  research  team  faced  the  same  proliferation  of 
plans  in  this  section  as  in  the  last.    Since  across-the- 
board  assessments  of  final  output  were  impossible,  we 
decided  to  concentrate  on  the  priority  areas.  Each 
ESED  was  asked  to  provide  us  with  a  current  status  report 
on  their  priority- level  plans. 


The  Division  of  Languages  in  ESED  #1  (which 
includes  the  reading  priority)  clearly  states  who 
approved  the  divisional  plan. 
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Unfortunately,  ESED  #1  gave  us  a  status  report  in 
Early  Childhood  Education  v/hile  SSED  #2  refused  to  give 
us  any  at  all.^^    n^i^e  from  the  fact  that  the  Early 
Childhood  project  reports  that  its  programs  are  on 
schedule  and  that  most  of  what  it  set  out  to  accomplish 
has  been  accomplished^  not  much  detail  can  be  added^  We 
cannot  generalize  beyond  Early  Childhood  to  other  pro- 
grams in  rSED  #1;  we  obviously  cannot  do  so  in  ESED  #2. 

This  data  gap  is  regrettable  given  the  importance 
of  measurable  progress  as  an  index  of  MAA  program  quality. 
But  the  attitudes  expressed  about  objective  progress 
offer  some  subjective  evidence. 


Section  3:    SEP  Attitudinal  Data 


Plans  and  progress  toward  them  can  also  be  as- 
sessed attitudinally.    VThat  do  people  inside  the  organi- 
zation feel  about  the  plans  with  which  they  work?  How 
far  do  these  individuals  think  the  SED  has  gone  in  attain- 
ing their  objectives?    What  has  been  done,  in  their 
opinion,  to  produce  operable  plans  and  transform  them 
into  genuine  educational  benefits? 

The  research  team  organized  the  data  relevant  to 
these  issues  into  two  general  categories: 

A.  Development  of  a  long-range  strategic  plan 

B.  Progress  toward  Goal  Achievement 

Content  categories  and  questionnaire  items  measured 
individual  beliefs  on  these  crucial  matters.    We  have 
already  compared  selected  plans  with  established  criteria 
and  have  attempted,  albeit  unsuccessfully,  to  report  on 
progress  toward  attainment  of  objectives.    No  matter 
what  the  research  team  concludes,  however,  unloso  people 
within  the  organization  believe  that  a  workable  plan  has 
been  written  and  that  progress  has  been  made  tov;ard  it, 
the  life  expectancy  of  the  plan  is  short. 

The  method  of  presentation  of  content  and  question- 
naire data  is  identical  to  that  followed  in  Chapters  Four 


Responding  to  our  request  for  a  report  on  current 
progress  toward  objectives,  the  Coordinator  of  Planning 
wrote  that  "the  Executive  Staff  will  be  unable  to  devote 
the  time  and  energy  required  to  complete  the  multi-page 
report.    General  reaction  across  the  Department  is 
basically  unfavorable  to  additional  non-productive  work 
of  this  type"  (24  April  1972). 
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and  Five;  the  research  team  sees  no  compelling  need  to 
re-introduce  them  here*    Any  exceptions  are  clearly 
marked. 


A.    Development  of  a  Long-Range  Strategic  Plan 

1.  Produce  and  implement  a  long-range  strategic 
plan.       "  ' 

Interview  Question:    What  do  you  feel  you 
will  obtain  (have  obtained)  frcxn  MA 
training? 

Range  of  Scale  Possibilities:     (1)  no  value 
to  (7)  maximum  value. 

States:    El^  E2. 

Points  of  Times     Tl,  T2,  T3,  T4. 

2.  My  organization's  overall  plan  is  operable. 
Points  of  Time:     Tl,  T2,  T3,  T4. 
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Item  _1  :    Produce  and  Implement  a  Long  Range  Plan 


Fall.  1970     \  Spring.  1971 

Fall.  1971 

Spring,  1972 

El  5  E2 

1  2 

El     5  E2- 

^1     ^  ^2 

N 
5 

N 

10 

N 

10 

N 

10 

N  N 
7  8 

N 
7 

N 
9 

Kruska 

1-Wallis  One-Waj 

r  Analysis  of  Variance 

H=  0.633 
Sie.=  NS 

H=  0.321 
Sie.=  NS 

H=0.0134 
Sifi.=  NS 

H=  2.868 
Sig.=  NS 

Binomial  Test  of  Proportions 

P=0.002 
Sig.=  .01 

P=  .762 
.   Sig.=  NS 

P=  .872 
.  SiK.=  NS 

P=0.430 
SiK.=  NS 

Fall  1970        to              Soring  1972 

E,            §  E^ 

N 
5 

N 
7 

N 

10 

N 

9 

Kruska. 

L-Wallis  One-Way  Analysis  of  Variance 

H=  0.5340 

H=  10.1400 
Sie.=.01 

Binomial  Test  of  Proportions 

P=  0.005 
Sig.=  .01 

P=  0.10 
Sig.=  NS 

398 


CM 


60 

c 
u 

C/3 


00 


o 


00 
CO  • 


in 
in 


o 


00 
iH 

CO  • 


in 


in 


'00 


.CO 


CO 


Q  ' 

CO 


in 


in  vo 


in 
o 

CM 


CM 


00 
00 


fH  CM 
CO  C/) 


iH  CM 
»-i  %  :|fc 
cd  Q  Q 
4^  M  U) 
C  C/)  CO 


6   B       rH  cd 


-         p  O 

>4  M  N  M 

Q<  Pi  6  C 
K  X  ,0  O 

u)  w  u  a 


H 

usr 


usr 


cisr 


O 

(/) 
(/) 
fH 

CO 

K 


.11 


CO 


CO 


CM 


<4H 
•H 
CO 

•H 

CO 


•H    ^  rH 


Q  H 

CO 
O 

fH 

cd  CO 

0)  o 
U3 


O  , 


41: 
Q  CM 

CO 

Q 
iH  U 
cd  CO 

C  iH 

o  o 

6  U 
0)  o 


CN 
CN 


m  o 


0  c  o 


Q 


CM 
% 
Q  r- 

CO 


Q  (M 
CO 


iH  tt  fH  u; 
cd  c/:  cd 


O  0 

6  V 


d)  0 

OiCJ 


399 


Item  #1  concerns  the  attitudes  of  ton  management 
personnel  in  the  two  SEDs  toward  their  pa  it  AMA  train- 
ing*   Specifically  we  were  interested  in  whether  they 
felt  their  agencies  were  producing  and  implGmenting  a 
long  range  plan  (the  fourteenth  training  goal  originally 
listed  by  tne  Mili)  • 

An  emalysis  of  their  responses  over  the  eighteen 
months  of  this  evaluation  project  revealed  no  significant 
differences  between  the  states  in  the  degree  of  emphasis 
given  this  item.    There  was  a  significant  difference 
of  awareness  prior  to  training  between  El  and  E2;  more 
top  managers  expected  to  develop  a  long-range  plan  in 
E2  than  was  the  case  in  El. 

A  comparison  within  the  states  does  reveal  a 
significantly  negative  training  effect  in  State  E2. 
Expectations  of  managers  in  that  state  agency  were 
frustrated  when  the  plan  they  expected  to  produce  as  a 
result  of  AMA  training  did  not  materialize  as  they  had 
hoped. 

State  El  showed  no  significant  training  effects. 
There,  managers'  expectations  regarding  a  long-range 
plan,  low  to  begin  with,  were  not  disappointed* 

Item  #2;    Fall  1970  to  Spring  1971 

My  organization's  overall  plan  is  operaible  was 
asked  over  the  entire  eighteen  months  of  this  project. 
A  T1-T2  comparison  reveals  negative  training  effects  in 
both  SED's.    State  El  and  State  E2  both  experienced 
declines  in  the  degree  to  which  personnel  of  th  se 
departments  felt  they  had  workable  plans*    Their  de- 
clines were  sharper  than  the  one  shown  by  the  Control 
State.    Both  states,  then,  were  significantly  less  cer- 
tain of  the  feasibility  of  their  plans  after  AMA  train- 
ing than  before  it. 

Item  #2;    Spring  1971  to  Fall  1971 

In  this  comparison,  a  reversal  of  the  trend  that 
developed  over  the  immediate  pre/post-training  periods 
occurred.    All  the  SED's  experienced  a  resurgence  in 
the  extent  to  which  people  believed  in  their  plans.  El 
and  E2  developed  a  greater  degree  of  stabilization  over 
time  than  did  the  Control  State,  and  consequently  dis- 
played a  positive  training  effect. 

Item  »2;    Fall  1971  to  Spring  1972 

Over  this  period,  all  the  states  fell  off  in  the 
degree  to  which  their  plans  were  seen  as  operable,  but 
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in  no  case  did  they  drop  to  thoir  T2  levels.    El  dif- 
fered significantly  from  both  Control  1  and  Control  2 
(holding  time  constant) f  but  El  evidenced  no  signifi- 
cant change  over  time.    State  E2  registered  a  negative 
training  effect.    Personnel  thare  lost  faith  in  their 
plans  much  faster  than  the  staff  of  Control  State  1. 
This  also  produced  a  significant  difference  between 
State  E2  and  Control  1  when  time  was  held  constant. 

Item  »2!    Fall  1970  to  Spring  1972 

Prom  the  longitudinal  perspective  of  this  report, 
AMA  training  had  no  effect  on  the  attitudes  of  Experi- 
mental States'  personnel  toward  their  plans.    By  Spring 
1972  State  El  and  E2  had  both  returned  to  their  approxi- 
mate pre-training  attitudinal  levels.    State  E2  con- 
tinued to  show  a  significant  disparity  between  itself 
and  Control  State  1,  holding  time  constant,  but  no  change 
occurred  over  time  between  these  two  states.    Thus  no 
training  effect  is  shown. 


DATA  SWIHARY 

Development  of  a  Long- 
Range  Strategic  Plan 

Fall,  1970    -    Spring,  1972 

Type  of  Data 
Item  CONTENT 

1  Produce  and  implement 
a  long-range  strategic 
plan. 

QUESTIONNAIRE 

2  My  organization's  over- 
all plan  is  operable. 


IflPACT  OF  TRAINING 


Positive 
Effect 


No  Negative 
Effect  Effect 


El 


E2 


E1,E2 


The  two  items  designed  to  measure  attitudes  toward 
this  AMA  training  goal  indicate  that,  for  the  most  part, 
AMA  training  did  not  affect  staff  attitudes  about  pro- 
ducing a  feasible  plan  for  their  agencies. 

The  one  effect  recorded  here  indicated  that  the 
managers  in  State  E2  felt  worse  about  their  agency's 
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overall  plan  after  training  than  before. 

Those  two  items  indicate  that^  from  a  longitudinal 
perspective^  personnel  of  both  states  have  not  seen  any 
evidence  to  increase  their  enthusiasm  for  their  agencies* 
overall  plans.    Managers  in  E2  have  apparently  found 
reason  to  decrease  their  faith  in  the  agency  plan. 


B.    Progress  Toward  Goal  Achiev€m\ent 

1.  Number  of  objectives  toward  which  progress 
has  been  made. 

Range  of  Scale  Possibilities:     (1)  no  objec- 
tives to  (7)  many  objectives. 

2.  Level  of  progress  toward  those  objectives* 

Range  of  Scale  Possibilities:     (1)  no  pro- 
gress to  (7)  much  progress • 

Both  these  items  were  based  on  the  following: 

Interview  Question:    Toward  what  action  plan 
objectives  has  measurable  progress  been 
made  by  your  division? 

States:     El,  E2,  CI. 

Points  of  Time:    T3^  T4. 

The  questionnaire  item  is: 

3.  As  I  see  it,  my  organization  has  made  pro- 
gress in  attaining  its  objectives. 


Points  of  Time:     T3,  T4. 
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Item  1_:    Number  of  objectives  toward  which  Progress 

has  been  made. 


F 

111.  1971 

Sp 

ring.  1972 

El  8Ci 

Ej  8  Cj 

8 

E,  8Cj 

E2  8Ci 

Ej  8  E2 

N 

9 

N 

11 

N 

10 

N 

11 

N 

9 

N 

11 

N 

9 

N 

11 

N 

10 

N 

11 

N 

9 

N 
10 

Kruskal-Wal. 

Lis  One-Wa] 

f  Analysis  of  Variance 

H«  1.213 
Sig.«  NS 

.H-0.019 
Sig.»NS 

H» 1.213 
Sig.-  NS 

H«3.753 
Sig.«NS 

H'0.044 
Sig.=  NS 

H=3.081 
Sig.=  NS 

B: 

Lnomial  Test  of  Proportions 

?♦ 1.000 
Sig.«NS 

P=l .  000 
Sig.»NS 

P= 1.000 
Sig.»NS 

P=1.000 
Sig.'NS 

P=l .  000 
.Sig.=NS 

P= 1.000 
.Sig.=  NS 

Fall, 

1971 

to  Spring 

,  1972 

El  a 

^1 

E2  a 

H 

Cl 

8  Cj 

N 

N 

N 

N 

N 

N 

9 

9 

10 

10 

11 

11 

Kniskal-Wallis  One-Way  Analysis  of  Variance 


H=  0.779 

H-0.005 

H=0.052 

Sig.«=  NS 

,Sig.=  NS 

,Sig.  =  NS 

Binomial  Test  of  Proportions 


P=  1.000 

P= 1.000 

P=1.000 

Sig.»  NS 

Sig.«  NS 

Sig.»NS 
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Itea  2    ;    Level  of  Progress  toward  those  Objectives. 


F 

111.  1971 

Sp 

ring.  1972 

Ej  9  Ej 

E,  »Cj 

h  ^  ^1 

El  5  h 

N 

9 

N 

11 

N 

10 

N 

11 

N 

9 

N 

11 

N 

9 

N 

11 

N 

10 

N 

11 

N 

9 

N 

10 

Kruskal-Wai: 

Lis  One-Wa^ 

/  Analysis  of  Variance 

H- 1.213 
Sig.-  NS 

H-0.447 
Sig.-  NS 

H- 1.213 
Sig.-  NS 

H-4.688 
Sig--  .05 

""0.100 
Sig.-  NS 

"=4.335 
Si8-=  .05 

B: 

momial  Test  of  Prop 

ortions 

P*1.000 
Sig.-  NS 

P=1.000 
Sig.-NS 

P= 1.000 
Sig.-  NS 

P=1.000 
Sig.»NS 

P=  1.000 
.Sig.=  NS 

P'1.000 
,Sig.  =  NS 

Fall, 

1971 

to  Spring 

.  1972 

Ej  S  Ej 

Ej  S  Ej 

N 

9 

N 
9 

N 

10 

N 
10 

N   1  N 
ll|  11 

Kruskal-Wallis  One-Way  Analysis  of  Variance 


H=  1,996 
Sig.«  NS 


H-1 . 000 


Sig. 


NS 


H=  0.10' 
Sig.=NS 


Binomial  Test  of  Proportions 


P=1.000 

P=l .  000 

P=1.000 

Sig.o  NS 

Sig.»  NS 

Sig.-NS 
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Item 


As  I  see  ih.  mv  oraaniy.afinn  ]^r,f,  made 
progress  in  attaining  its  ob-ieGt-ives . 


Fall,  1971 

Spring,  1972 

Experinental  SED#1 

N 

68 

X 

5.411 

SD 

1.340 

N 

■  40 

X 

5.325 

SD 
0.888 

E]q[>erlBental  SED*2 

45 

4.866 

1.341 

"40 

4.7-)5 

"1.349 

Control  SED  *1 

§7 

5.253 

1.146 

61 

5.114 

1.239 

Control  SED  *2 

61 

5.295 

§6 

5.090 

1.063 

Total 

24] 

207 

IVo  Nay  Analysis 
Variance 


of 


Experinental  SED  # 1 
W/Control  SED  «1 
Experinental  SED  #1 
w/Control  SED  «  2 
Eiqperinental  SED  #2 
w/Control  SED  «1 
Experinental  SED  #2 
w/Control  SED  92 


^3*^4 


:oi. 


Bow 


Col. 


tow 


Col. 


tow 


:oi. 


low 


0.504 


0.859 


1.248 


0.430 


4.271 


0.726 


4.596 


NS 


NS 


NS 


NS 


NS 


,05 


NS 


,05 
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Measurable  progress  toward  objectives  has  been 
stressed  here  as  an  important  index  of  the  effectiveness 
of  AMA  training.     In  Fall  1971  and  again  in  Spring  1972, 
we  asked  top  managers  in  each  state  what  action  plan 
objectives  they  had  made  measurable  progress  toward.  We 
were  interested  in  the  nxamber  of  objectives  they  mentioned' 
as  well  as  the  level  of  progress  indicated,  and  antici- 
pated both  indices  would  increase  with  time. 


Item  #1  concerns  the 
which  progress  has  been  made 
managers.  An  analysis  of  th 
between  the  Experimental  Str. 
ing  the  two  points  in  time  u 
asked.  No  statistical  diffe 
the  Experimental  SED  and  the 
Experimental  SED's  themselve 
within  the  states  also  shows 
year  after  training,  no  real 
of  the  number  of  objectives 
made. 


number  of  objectives  toward 
t  in  the  opinion  of  the  top 
is  item  shows  no  change 
tes  and  the  Control  SED  dur- 

which  this  question  was 
rences  exist  either  between 

Control  or  between  the 
s.    A  comparison  over  time 
no  differences.    During  the 
movement  occurred  in  terms 
toward  which  progress  was 


Item  #2  is  concerned  with  the  extent  to  which  those 
objectives  (on  which  progress  has  been  made)  have  been 
actually  achieved.     Only  one  significant  difference 
emerges  between  the  states.     In  the  T4  comparison  between 
El  and  E2,  managers  in  El  felt  they  had  made  more  progress 
toward  their  objectives  than  did  managers  in  E2.  No 
training  effects  are  indicated  in  this  data,  however,  as 
none  of  the  states  significantly  changed  over  time  regard- 
ing the  level  of  progress  they  were  making. 

Item  #3;     Fall  1971  to  Spring  1972 

Organizational  attitudes  toward  Item  #3,  As  I  see 
it,  my  organization  has  made  progress  in  attaining  its 
objectives,  is  an  important  component  in  evaluai:ing  pl;^n 
effectiveness.    While  E2  does  not  report  any  increases 
or  decreases  in  the  extent  to  which  their  objectives 
have  been  attained  between  Fall  1971  and  Spring  1972, 
they  have  made  less  progress  than  either  Control  State. 
No  difference  exists  between  El  and  C1/C2  nor  were  any 
changes  registered  over  time. 


ERIC 
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DATA  SUMI^ARY 


Progress  Toward 
Goal  Achievement 

Fall,  1970  -  Spring,  1972 


Type  of  Data 
Item  CONTENT 

1  Ntunber  of  objectives 
toward  which  progress 
has  been  made 

2  Level  of  progress 
toward  those  objectives 


IMPACT  OF  TRAINING 


Positive 
Effect 


No  Negative 
Effect  Effect 


E1,E2 
E1,E2 


As 


QUESTIONNAIRE 
I  see  it,  my  organi- 


zation has  made  progress 
in  attaining  its  objec- 
tives • 


E1,E2 


The  two  interview  Content  items  and  the  question- 
naire item  all  indicated  no  training  effects  in  either 
Experimental  State.    While  State  El  made  more  progress 
than  State  E2f  this  difference  camnot  be  attributed  to 
AtlA  training  because  no  statistically  significant 
change  occurred  over  time. 

Expecting  some  change  in  the  course  of  the  year, 
the  research  team  asked  "Toward  what  action  plan  objec- 
tives has  measurable  progress  been  made  by  your  division?" 
The  obvious  lack  of  movement  seemed  to  be  a  direct  result 
of  the  respondents'  inability  to  point  to  any  measurable 
progress  concerning  goal  achievement.    Most  respondents 
thought  that  they  had  moved  ahead,  but  few  could  specify 
any  specific,  measursO^le  indications  of  progress. 
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DATA  SUMTIARY 

End  Result  Variables  - 
State  Education  Denartitiont 

Fall,  1970   -  Spring,  1972 

IMPACT  OP  TRAINING 

Positive       No  Negativo 
Item        Type  of  Data  Effect      Effect  Fffect 

DEVELOPMENT  OF  A  LONG-RANGE  STRATEGIC  PLAN 
CONTENT 

1  Produce  and  implement  a 

long-range  strategic  plan  El  E2 

QUESTIONNAIRE 

2  My  organization's  overall 

plan  is  operable.  E1,E2 

PROGRESS  TOWARD  GOAL  ACHIE^TEMENT 
CONTENT 

1  Number  of  objectives  toward 
which  progress  has  been 

made  E1,E2 

2  Level  of  progress  toward 

those  objectives  E1,E2 

QUESTIONNAIRE 

3  As  I  see  it,  my  organiza- 
tion has  made  progress  in 

attaining  its  objectives •  E1,E2 


The  five  items  designed  to  measure  respondents* 
attitudes  in  the  SED's  toward  their  overall  plans  as  well 
as  toward  progress  made  in  achieving  the  objectives  laid 
down  in  those  plans,  indicate  that  the  AJIA  training  pro- 
gram had  no  positive  impact  on  attitudes  about  these 
items.    The  one  effect  that  was  recorded  showed  a  nega- 
tive training  effect  in  State  E2,  whose  expectations 
about  the  development  of  a  long-range  plan  wore  frus- 
trated.   Fewer  managers  mentioned  this  as  an  outcome  of 
AMA  training  in  the  Spring  of  1972  than  had  originally 
(before  training)  expected  such  planning. 

All  other  items  in  this  category  displayed  no 
training  effects.    Managers  viewed  their  progress  as 
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being  basically  unaffected  over  the  course  of  this 
evaluation*    They  wore  making  no  more,  nor  any  less 
progress  than  they  had  been  earlier* 

As  we  mentioned  above,  one  clear  reason  for  this 
lack  of  apparent  change  in  goal  achievement  seems  based 
in  the  fact  that  few  managers  have  developed  adequate 
evaluation  strategies*    Thus  they  cannot  be  sure  to  what 
extent  they  are  making  progress* 


Section  4:    The  Experimental  LEAs 

This  section  assesses  the  effect  of  AMA  training 
on  LEA  end- result  varieibles*    We  evaluate  the  actions  taken 
and  the  plans  produced  by  the  LEAs  after  training*  We 
seek  to  determine  the  extent  to  which  these  plans  corres- 
pond to  the  criteria  for  a  satisfactory  Plan  that  the 
ESEDs  used*    Secondly,  we  are  interested  in  attitudes 
expressed  by  participants  about  end  result  variables  * 
Due  to  lack  of  available  data  from  LEAs,  it  will  not  be 
possible  to  systematically  study  progress  toward  ob^ec^ 
tives  *    But  the  two  groups  o^  data  we  do  possess  offer 
sufficient  evidence  on  which  to  base  some  conclusions* 


A*  Action 

To  examine  the  plans,  we  asked  the  four  (4) 
Bxperimental  LEA's  to  forward  current  copies  to  us* 
ELEA's  #1,  #2,  and  #4  complied  with  this  request;  ELBA 
#3  did  not*^^    Given  the  importance  of  actual  output  after 
training,  this  loss  of  data  is  unfortunate. 

However,  using  the  plans  we  were  given,  we  are 
ablo  to  compare  each  of  them  to  the  criteria  for  analysis* 
Each  plan  was  compared  with  the  criteria,  not  with  other 
plans*    Hence  no  direct  coinparisons  should  be  made  between 
the  plans  themselves;  because  of  unlisted  gradations 
within  the  *yes"  and  "no**  categories,  no  comparisons  can 
be  made*    A  "yes"  in  one  category  may  signify  complete 


ELEA  #1  sent  a  photocopy  of  their  AMA  plan  as 
their  "current"  plan*    The  AHA  plan  was  not  very  readable; 
it  reflected  an  initial  attempt  at  systematic  planning 
and  needed  further  refinement    and  editing*    That  the  14- 
month-old  AMA  plan  is  still  regarded  as  the  current  plan 
indicates  that  little  evolution  in  the  planning  process 
has  occurred* 
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success  while  a  "yes"  in  another  category  may  identify 
a  borderline  case.    Tho  plan  with  the  most  "yes"  scores 
is  not  necessarily  the  best,  and  vice  versa. 


ANALYSIS  OF  ELBA  PLANS 
Spring,  1972 


ELEA  #1 

ELEA  #2 

ELEA  «3 

Criteria 

A  N 

A  L  Y  S  I  S 

1 

Title 

Yes 

Yes 

Yes 

2 

Identification 

No 

No 

No 

3 

Needs  Assessment 

No 

No 

No 

4 

Recommendations 

Yes 

Yes 

Yes 

5 

Expected  Results 

Yes 

Yes 

No 

6 

Time  Frames 

Yes 

Yes 

No 

7 

Responsibi lity 

No 

No 

No 

8 

Resource  Requirements 

No 

Yes 

No 

9 

Evaluation  Strategy 

No 

No 

No 

10 

Rationale 

No 

No 

No 

Yes=4 

Yes=5 

Yes=2 

No  a6 

No  =5 

No  =8 

Comments 

The  LEA  plans  were  fairly  consistent  in  their 
strengths  and  weaknesses.    All  demonstrated  an  ability  to 
write  behavioral  objectives  in  a  format  with  expected 
results  and  time  frames;  however,  ELEA  #4's  second  draft 
plan  (used  to  construct  the  preceding  diagram)  met  these 
criteria  less  well  than  did  its  first  draft. 15 


ELEA  #4  gave  the  research  team  two  planning 
documents:    "Current  Planning  Process—First  Progress 
Report"  and  "Five  Year  Educational  Master  r'an — Second 
Draft."    The  first  followed  our  criteria  much  more  than 
the  second.    This  gradual  shift  av;ay  from  measurable 
objectives  parallels  a  similar  shift  by  the  State  Education 
Department  after  the  adverse  reaction  of  the  State  Board 
to  such  planning.    The  second  draft  offers  more  narrative 
explanations  than  the  first  but  is  more  difficult  to  under- 
stand.    As  the  latest  version  of  the  agency  plan,  the 
ERJC        second  draft  was  judged  against  our  criteria. 
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Resource  requirements  and  responsibility  for 
implomentation  were  partially  incorporate(2  into  these 
plans,  but  not  nearly  enough  to  qualify  as  satisfactory. 
ELBA  #2  attached  more  price  tags  while  identifying 
fewer  of  the  personnel  responsible  for  accomplishment  of 
goals. 

Explanation  of  the  need  and  rationale  for  various 
objectives/strategies  as  well  as  evaluation  methods 
were  particularly  weak.    Where  suggested  evaluations  did 
appear,  they  were  usually  listed  apart  from  the  objectives 
they  might  have  evaluated.    To  some  extent  this  could  be 
attributed  to  the  absence  of  appropriate  measures,  al- 
though some  evaluations  were  planned  for  some  objectives 
in  the  future. 

Overall,  plans  do  not  meet  acceptable  standards  on 
as  many  criteria  as  they  should.    The  addition  of  needs 
assessments,  resource  requirements,  and  evaluation 
strategies  would  improve  them*    Greater  attention  should 
also  be  paid  to.  listing  the  personnel  responsible  for 
various  parts  of  the  plan. 


B.  Attitudes 

Turning  to  attitudinal  evaluation,  we  assess  the 
same  basic  categories  for  the  LEA*s  that  were  used  for 
the  SED's: 

A.  Development  of  a  Long-Range  Strategic  Plan 

B.  Progress  Toward  Goal  Attainment 

Since  interviews  were  not  conducted  in  the  LEA's  in  con- 
nection with  this  report,  it  is  impossible  to  present 
Content  data.    Consequently,  fewer  items  are  associated 
with  each  category  than  was  the  case  in  analyzing  the 
SED's. 

Each  item  is  displayed  and  explained,  however,  in 
a  similar  style;  we  need  not  repeat  the  elements  in  our 
examination.    Again,  any  exceptions  to  what  has  gone 
before  will  be  indicated. 


End-Result  Variables 

A.    Development  of  a  Long-Range  Strategic  Plan 

The  questionnaire  item  is; 

1.    My  organization's  overall  plan  is  operable. 


Points  of  Time:    Tl,  T2,  T3,  T4. 
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Item  #1;     Fall  1970  to  Spring  1971 

Item  #1,  My  organization's  overall  plan  is  operable^ 
was  relevant  to  the  four  Experimental  LEA*s  between  Fall — 
1970  and  Spring  1971,  a  period  including  the  pre-  and 
post-training  intervals.    After  spending  several  weeks  at 
Hamilton  to  learn  the  team  planning  process  and  develop 
an  action  plan  for  their  school  district,  ELI  and  EL2 
felt  their  plans  were  less  feasible  after  training  than 
before.    EL3  and  EL4  also  faltered  in  their  opinion  of 
current  plans,  but  not  substantially.     Consequently,  no 
training  effects  were  assigned  to  ELS  and  EL4  but  the 
diminished  felt  practicability  in  ELI  and  EL2  was  suffi- 
cient to  be  due  to  negative  effects  of  training.  Although 
we  are  confident  in  these  judgments,  our  conclusions  would 
have  been  strengthened  here  by  comparing  scores  in  Experi- 
mental groups  with  Control  group  scores. 

Item  #1;     Spring  1971  to  Fall  1971 

The  negative  training  effects  assigned  T1-T2  were 
transformed  into  positive  effects  T2-T3  in  the  cases  of 
ELI  and  EL2.     Both  organizational  plans  were  much  improved 
during  1971 — so  much  so  that  AMA  credited  with  these 
increases.     Gains  recorded  by  ELS  and  t'L4  were  insignifi- 
cant, and  training  effects  were  not  ascribed  to  either 
organization. 


Item  #li     Fall  1971  to  Spring  1972 

Analysis  of  changes  ?nd  differences  between  Fall 
1971  and  Spring  1972  was  validated  by  the  presence  of 
Control  groups.     Their  addition  made  possible  a  more 
unambiguous  analysis  of  training  effects.    ELI  felt  better 
about  their  educational  plans  than  either  EL2  or  the  Con- 
trol group;  but  this  difference  was  due  more  to  ELl's 
superiority  remaining  constant  over  tim^^,  than  to  signifi- 
cant changes  in  ELI,  EL2,  or  CLl.    No  other  changes  or 
differences  were  recorded.     The  AMA  training  program, 
therefore,  had  no  effect  during  this  period. 

Item  #1:     Fall  1970  to  Spring  1972 

The  main  advantage  of  a  T1-T4  comparison  was  that 
it  permitted  a  longitudinal  evaluation  of  training  impact. 
Negative  or  positive  effects  observed  in  interim  periods 
may  fade,  or  grow  stronger,  with  time.     Some  effects  may 
not  appear  until  a  year  or  two  after  training.     In  this 
case,  the  long  view  made  it  possible  for  us  to  hold  that 
AMA  exerted  positive  effects  in  ELI  and  EL2  but  none  at 
all  in  EL3  and  EL4.     Plans  were  considered  significantly 
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more  operational  in  ELI  and  EL2  in  Spring  1972  than  they 
had  been  in  the  Fall,  before  training.    A  comparable 
increase  did  not  occur  in  EL3  and  EL4. 


DATA  SUMMARY 

Development  of  a  Long-Range 
Strategic  Plan 

Fall,  1970  -  Spring^  1972 


IMPACT  OF  TRAINING 


Type  of  Data 


Positive 
Effect 


Negative 
Effect 


Item 


QOESTIONimRE 


No 
Effect 


1 


My  organization's  overall 
plan  is  operable. 


EL1,EL2 


EL3,EL4 


B.    Progress  Toward  Goal  Achievement 


The  questionnaire  item  is: 


1. 


As  I  see 


it#  roy  organization  has  made 


in  attaining  its  objectives. 


progress 


Points  of  Time:    T3/  T4. 
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Item      1       As  t  see  it,  my  organization  has 
made  progress  in  attaining  its 
objectives. 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 

N 

X 

SD 

N 

X 

SD 

32 

6.125 

0.941 

31 

5.903 

0.789 

51 

^.463' 

1.091 

.  32 

•^-375 

1.070 

24 

5.750 

10.793 

23 

5.478 

0.947 

,.  19 

5.241 

0.950 

?J 

4.fl14 

1.210 

30 

R.733 

0.944 

20 

n.7>;3 

,lfi 

i.nfi 

_  3^ 

5.303 

1.074 

182 

166 

Two  Way  Analysis  of 
Variance 

T3  T4 

Experimental  LEA#1W/ 
Experimental  LEA#2 

F 

Sijtnif. 

Coll 

0.894 

NS 

Row 

11.182 

.001 

Experimental  LEA#SW/ 
Experimental  LEA#4 

CpJ, 

3.156 

NS 

Row 

8.892 

.01 

lExperr.mental  LEA#1W/ 
pontrol  LEA  #1 

Col. 

1.129 

NS 

Row 

4.324 

.05 

Experimental  LEA#2W/ 
Control  LEA  #1 

Col. 

0.403 

NS 

Row 

1.548 

NS 

Experimental  LEA#3W/ 
Control  LEA  #2 

Col. 

0.512 

NS 

Row 

NS 

Experimental  LEA#4"w7 
Control  LEA  #2 

Col. 

l.iy3 

MS 

Row 

1.978 

MS 
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Item  #1;     Fall  1971  to  Spring  1972 

This  question  was  posed  m  Y2  only.     Our  purpose 
was  to  gain  some  perspective  on  how  personnel  in  the  LEA's 
felt  about  the  progress  they  were  making  toward  their 
goals. 

Item  #1,  As  I  see  it^  my  organization  has  made 
progress  in  attaining  its  obpectives^  recorded  no 
significant  effects  that  can  be  attributed  to  AMA  train- 
ing.    State  ELI  and  EL2  significantly  differed  from  each 
other,  holding  time  constant;  FLl  reached  a  higher  level 
of  progress  than  EL2^  but  no  .    jnificant  changes  regis- 
tered over  time.     ELI  was  also  significantly  different 
from  Control  ELI,  holding  time  constant. 

EL3  and  EL4  differed  from  each  other.     Holding  time 
constant^  ELS  ranked  higher  at  both  points  in  time^  but 
both  groups  made  no  change  over  time. 

In  summary,  no  LEA  experienced  any  significant 
change  over  time  concerning  its  attitudes  toward  progress 
on  objectives.    All  felt  they  were  approximately  at  the 
same  level  of  achievement  in  the  Spring  of  1972  as  they 
had  been  in  the  Fall  of  1971.     An  examination  of  mean 
scores  indicates  that  all  the  LEA's  actually  experienced 
slight  declines  over  this  period  of  time,  although  these 
were  never  statistically  significant. 


DATA  SUMMARY 


Progress  Toward 
Goal  Achievement 

Fall,  .1970  -  Spring,  1972 


Type  of  Data 

Item  '  QUESTIONNAIRE 

1  As  I  see  it,  my  organiza- 
tion has  made  progress  in 
attaining  its  objectives. 


IMPACT  OF  TRAINING 

Positive      No  Negative 
Effect    Effect  Effect 


EL1,EL2, 
EL3,EL4 
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DATA  SU!1MA.RY 
End  Result  Variables  -  LEA 
Fall,  1970  -  Spring^  1972 

IIIPACT  OF  TRAINING 


Positive        No  Negative 
Item        Type  of  Data  Effect       Effect  Effect 

DEVELOPMENT  OF  A  LONG-RANGE  STRATEGIC  PLAN 

QUESTIONNAIRE 

1     Ily  organization's  ovorall 

plan  is  operable.  EL1,EL2  EL3,EL4 

PROGRESS  TOWARD  GOAL  ACHIEVEMENT 

QUESTIONNAIRE 

1     As  I  see  it,  my  organi- 
zation has  made  progress 

in  attaining  its  objec-  EL1,EL2, 
tivGS.  EL3,EL4 


Analysis  of  the  attitudinal  data  indicates  that 
the  AT^  program  had  little  impact  on  how  LEA  end-result 
variables  were  viewed. 

Positive  effects  were  recorded  in  regard  to  how 
operable  ELI  and  EL2  thouglit  their  organizational  plans. 
The  plans  were  considered  more  feasible  in  Spring  1972 
than  in  Fall  1970,  before  the  start  of  training. 

But  this  opinion  v;as  not  shared  by  EL3  and  EL4, 
both  of  whom  saw  no  change  in  their  plans'  practicality. 
None  of  the  LEA's  perceived  increased  progress  toward 
objectives  as  a  product  of  AI4A  training. 

In  fact,  during  the  first  two  periods  when  progress 
might  reasonably  have  been  expected  (T3  and  T4),  all  four 
lea's  sensed  less  progress  in  T4  than  in  T3.    This  decline, 
while  not  statistically  significant,  did  not  augur  well 
for  future  progress. 
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Section  5:    Summary  and  Conclusions 

It  is  easier  to  list  what  education  invests  in  its 
programs  than  to  measure  what  it  achieves  through  them. 
Discovering  whether  planning  guidelines  exist  or  whether 
performance  stemdards  have  been  developed  is  a  compara- 
tively concrete  problem.    Even  diagnosing  the  internal 
health  of  an  organization  can  be  achieved  by  a  longitudinal 
research  design  with  reliable  instruments.    But  it  is  more 
difficult  to  decide  whether  these  causal  and  intervening 
variables  have  improved  the  implementation  of  planning 
or  hastened  educational  progress  (end-result  variables) . 

This  analysis  begins  with  two  advantages,  however. 
One  is  the  audit  of  the  quality  of  the  plans  produced, 
which  was  conducted  using  the  best  evaluative  criteria 
available.    The  second  is  the  expressed  attitudes  of 
persons  inside  these  organizations:    how  operable  are 
their  plans  and  hov;  much  headway  has  been  made  in  accomp- 
lishing them? 


ANALYSIS  OF  ESED  PRIORITY  AREA  PLANS 
Spring,  1972 

ESED  #1  ESED  #2 

Career  Early  Human 

Educa-    Read-    Child-    Rela-  Read 


tion 

ing 

hood 

tions 

ing 

Criteria 

ANA 

L  Y  S  I 

S 

1 

Title 

Yes 

Yes 

Yes 

Yes 

Yes 

2 

Identification 

No 

No 

No 

No 

Yes 

3 

Needs  7xssessment 

Yes 

Yes 

Yes 

Yes 

No 

4 

Recommendations 

No 

Yes 

Yes 

Yes 

Yes 

5 

Expected  Results 

Yes 

Yes 

Yes 

Yes 

Yes 

6 

Time  Frames 

Yes 

Yes 

Yes 

Yes 

Yes 

7 

Responsibility 

No 

No 

No 

No 

Yes 

8 

Resource  Require- 
ments 

No 

No 

No 

No 

Yes 

9 

Evaluation 
Strategies 

No 

Yes 

No 

No 

No 

10 

Rationale 

No 

No 

No 

No 

No 

Yes=4  Yes=6  Ye8«5  Yes=5  Yes=7 
No  =6      No  =4      No  =5    no  =5  No  =3 
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ANALYSIS  OF  ELBA  PLANS 
Spring,  1972 


ELBA  #1  ELBA  #2  ELBA  #4 


Criteria 

*»          «i    JU    i     Q    X  o 

1 

Title 

Yes 

Yes 

Yes 

2 

Identification 

Mo 

No 

No 

3 

Needs  Assessment 

No 

No 

No 

•1 

RecommGndations 

Yes 

Yes 

Yes 

5 

Expected  Results 

Yes 

Yes 

No 

6 

Time  Frames 

Yes 

Yes 

No 

7 

Responsibility 

No 

No 

No 

8 

Resource  Requirements 

No 

Yes 

No 

9 

Evaluation  Strategy 

Nc 

No 

No 

10 

Rationale 

No 

No 

No 

Yes«4 
No  «6 

Yes=5 
No  =»5 

Yos=2 
No  =8 

ERIC 
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/iTTITUDINAL  DATA  SUR'IARY 

End  Result  Variables  - 
State  Education  Department 

Fall,  1970  -  Spring,  1972 


IfgACT  OF  TRAINING 


Positive       No  Negative 
I^ero        Type  of  Data  Effect     Effect  Effect 

DEVELOPMENT  OF  A  LONG-RANGE  STRATEGIC  PLAN 
CONTENT 

1  Produce  and  implement  a 

long-range  strategic  plan  El  B2 

QUESTIONNAIRE 

2  My  organization's  overall 

plan  is  operable.  E1,E2 

PROGRESS  TOWARD  GOAL  ACHIEVEMENT 
CONTENT 

1  Number  of  objectives  toward 

which  progress  has  been  made  E1,E2 

2  Level  of  progress  toward 

those  objectives  E1,E2 

QUESTIONNAIRE 

3  As  I  see  it,  my  organization 
has  made  progress  in  attain- 
ing its  objectives.  E1,E2 
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DATA  suririARy 

End  Result  Variables  -  Local 
Education  Agencies 

Fall,  1970    —    Spring,  1972 

IMPACT  OF  TPJVINING 


Positive        No  Negative 
Item  Type  of  Data  Effect       Effect  Effect 

DEVELOPMENT  OF  A  LONG-RANGE  STRATEGIC  PLAN 

QUESTIONNAIPJ 

1    My  organization's  overall 

plan  is  operable.  EL1,EL2  EL3,EL4 

PROGRESS  TOWARD  GOAL  ACHIEVEMENT 

QUESTIONNAIRE 

1    As  I  see  it,  my  organiza- 
tion has  made  progress  in  EL1,EL2 
attaining  its  objectives.  EL3,EL4 


This  examination  of  the  status  of  the  Experimental 
State  and  Local  Educational  Agencies'  end-result  variables 
relates  actions  to  attitudes  separately  described  in  the 
body  of  this  chapter.    It  is  necessary  to  establish  this 
relationship  because  what  people  in  organizations  do 
often  depends  on  how  they  feel,  and  vice  versa. 

It  can  be  argued,  we  think,  that  this  reciprocal 
relation  obtains  hero.    The  "mixed  reviews"  given  the 
plans,  and  the  general  "no  effect"  rating  attributed  to 
AMA  training  as  summarized  on  the  preceding  charts,  are 
connected.    One  of  the  reasons  why  peonle  in  the  SED's 
and  LEA'S  felt  AMA  training  had  no  effect  on  Dlan 
feasibility  or  progress  v/as  that  they  felt  the  plans 
were  not  feasible  and  they  lacked  measures  of  progress. 
Plans  were  not  completely  thought  out;  certain  key  elements 
called  for  by  generally  accepted  planning  criteria  were 
absent.    A  partial,  overgeneralizcd  plan  is  not  operable 
nor  does  it  offer  much  hope  of  successful  implementation. 

The  plans  that  were  reviewed  did  contain  the  cen- 
tral planning  elements:    objectives  (ends)  and  some 


strategics  (means)  for  attaining  these  ends.    The  plans 
set  forth  highly  professional  statements  of  what  the 
agency  would  accomplish  as  well  as  how  and  when  they 
intended  to  accomplish  it.    Seme  attention  was  also 
given  to  who  would  benefit  from  the  accomplishment. 

Unfortunately,  why  received  only  passing  atten* 
tion.    Lists  of  assumptions  and  beliefs  do  not  satisfy 
the  need  to  define  precisely  the  problems  a  plan  pro- 
poses to  solve,  nor  does  it  suggest  why  these  problems 
are  selected  for  solution.    Such  rationalization  is  a 
necessary  preliminary  to  decisions  on  secondary  objectives 
and  strategies  dictated  by  the  overall  approach  to  the 
problem. 

Recommendations  logically  follow  from  assessed 
needs.       Needs  can  exist  without  reccMiunendations; 
indeed,  in  a  system  with  many  needs  and  few  resources, 
they  are  nearly  inevitc&ble*    A  budget  only  stretches  so 
far.    But  recommendations  without  stated  needs  make 
little  sense.    A  problem  whose  existence  is  not  acknowl- 
edged in  detail  cannot  be  solved— although  planners  have 
been  known  to  try.    MeJcing  recommendations  without  making 
clear  the  reasons  behind  them  implies  that  needs  are  being 
assumed,  not  proved. 

Clearly,  writers  of  some  plans  we  examined  knew 
their  needs,  but  this  knowledge  is  not  conveyed  by  the 
plans.    By  and  large,  the  plans  do  not  logically  progress 
from  assessed  need  to  expected  solution,  with  well* 
reasoned  intermediate  steps.    This  is  less  true  of  the 
state  agency  plems  we  reviewed,  but  even  there  the 
discussion  of  needs  was  often  framed  in  prose  with  only 
modest  empirical  evidence.    If  plans  do  not  carefully 
study  needs r  they  tend  to  becc»ne  statements  of  personal 
or  agency  conviction  and  shibboleth  rather  than  carefully 
defined  attacks  on  existing  problems.  Illustratively, 
we  cited  the  example  of  one  SED  that  intended  to  attain 
a  future  85%  reading  achievement  level  without  evidence 
as  to  present  levels. 

Secondly,  the  issues  of  who  will  do  it  and  how  much 
will  it  cost  were  not  satisfactorily  addressed.  Responsi- 
bility for  plan  elements  was  indefinite  in  a  few  of  the 
SED  and  LEA  plems  made  available  to  the  research  team. 
It  is  certainly  essential  to  designate  in  the  plan  who 
will  coordinate  efforts  toward  broad  agency-wide  objec- 
tives or  individual  classroom  tasks.    The  responsibility 


It  should  be  noted  here  that  state-wide  Needs 
Assessments  are  presently  being  conducter'  by  both  Experi- 
mental SED's,  which  should  do  much  to  correct  this  "gap" 
in  the  planning  documents. 
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of  everyone  frequently  becomes  the  responsibility  of 
no  one,    Assignments  made  to  ••staff"  or  "selected 
personnel"  are  not  really  assignments  at  all. 

The  amount  of  money,  personnel,  and  materials 
required  to  accomplish  these  objectives  were  also  inade- 
quately developed.    A  ''wish  list"  is  ordinarily  a  plan 
completely  devoid  of  reality;  some  plams  could  be  con- 
sidered, at  least  in  part,  as  wish  lists.    It  is  relatively 
easy  to  list  objectives;  it  is  not  so  simple  to  pay  for 
them.    "Budgets  should  support  plans, "1^  and  the  finished 
planning  document  must  contain  full  budgetary  information, 
for  the  plan  cannot  be  approved  by  those  in  authority 
nor  evaluated  in  a  financial  vacuum. 

Thirdly,  plans  did  not  propose  viable  evaluation 
mechanisms  for  measuring  progress  toward  stated  objectives. 
This  is  a  widespread  problem  for  education  (as  well  as 
other  fields)  tifiat  is  not  peculiar  to  the  SED's  and  LEA's* 
The  measurement  of  student  and  administrative  achievement 
lacks  both  adequate  indices  and  methodology  to  generate 
reliable  information. 

Absence  of  adequate  instrximents  may  also  help  to 
explain  the  absence  of  needs  assessments  and  problem 
definitions.    Tools  used  to  measure  progress  can  also 
define  problems;  they  keep  interpretations  consistc^nt 
and  analyses  statistically  reliable.    But  plans  cai.:iot 
offer  data  on  current  needs  which  they  do  not  have  and 
cannot  obtain. 

Expressed  organizational  attitudes  are  even  more 
problematic.    Generally,  plans  were  not  viewed  as  more 
operable  a  year  after  training  than  they  had  been  before 
training.    Progress  had  reached  no  higher  level  during 
this  same  time  span.    This  indicated  not  what  the  AMA 
program  had  done  to  the  Experimental  educational  agencies, 
but  rather  what    xt'  failed  to  do  for  them. 

Plans  were  not  seen  as  generally  operable  by  the 
personnel  in  these  agencies  because,  from  the  perspective 
of  the  criteria,  they  were  indeed  inopered^le.    As  demon- 
strated earlier,  planning  documents  were  incomplete. 
Administrators  face  a  formidable  task  in  trying  to  follow 
a  plan  that  does  not  specify  the  need  it  addresses,  the 
money  or  men  it  will  cost,  task  responsibility,  or  mode  of 
evaluation. 


An  operational  principle  taken  from  A  Plan  for 
Planning  Elementary  and  Secondary  Education,  SED  #1,  p.  9. 
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The  lack  of  reported  progress  is  associated  with 
this  lack  of  operability.    The  organizations  did  not 
develop  adequate  evaluation  techniques  to  measure  pro- 
gress ^  yet  progress  that  cannot  be  measured  cannot  be 
reported. 


Overall  Summary  on  the  Impact  of  AMA  Training  on  the 
SED/LEA  Intervening  and  End-Result  Variables 

Our  exeunination  of  the  data  on  intermediate  and 
end-result  output  variables  indicates  that  the  training 
progrcun  with  the  American  Memagement  Association  exerted 
a  mixed  effect  on  the  plans  produced,  and  no  effect  on 
the  aegree  to  which  people  in  the  Experimental  agencies 
saw  themselves  progressing  toward  their  objectives. 

The  plans  we  examined  represent  good  first  or 
second  drafts  in  an  ongoing  planning  process  that  demands 
continuous  revision  to  meet  developing  needs.  Changes 
in  the  plans  themselves  can  also  reflect  increasing 
proficiency  in  planning.    The  AMA  did  succeed  in  giving 
the  Experimental  agencies  basic  skills  in  some  prerequi- 
sites of  good  planning,  but  not  in  others.  Perhaps 
because  of  what  AMA  failed  to  do  and  what  remains  to  be 
done,  plans  are  not  considered  operable  or  moving  toward 
goal  accOTiplishment.    If  the  "secondary"  elements 
described  above  were  added,  perhaps  the  plans  could  be 
operationalized  and  progress  generated. 

Given  specific  needs,  task  assignments,  cost 
figures,  and  evaluation  techniques,  plans  still  may  not 
work.    We  only  suggest  that,  without  these  details, 
plans  cannot  work. 
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CHJiPTER  SEVEN 


OVERALL  SUMMARY  AND  CONCLUSIONS 


This  chapter  attempts  to  integrate  the  findings 
and  analysis  of  previous  chapters  into  a  set  of  conclu- 
sions on  the  overall  effectiveness  of  the  A?IA  program. 

It  begins  with  a  review  of  AMA  change  strategies 
and  the  theoretical  connections  that  tie  these  strategies 
to  ongoing  organizational  settings.    We  use  an  adapta- 
tion of  Likert's  causal/intervening/cnd-result  variables 
framework  to  establish  these  connections. 

Then  we  summarize  data  and  interpretations  from 
earlier  chapters  of  this  report  in  order  to  provide  a 
background  perspective  for  further  analysis. 

Finally,  we  integrate  all  our  conclusions  (previous 
ly  separated  by  causal,  intervening,  and  end  result 
variable  divisions)  into  a  series  of  general  conclusions 
regarding  the  impact  of  the  AHA  training  program  on  the 
Experimental  educational  agencies. 

To  these  ends.  Chapter  Seven  includes  the  follow- 
ing sections: 

1.  Z^erican  Management  Association  Strategies 
for  Change 

2.  Summaries  and  Interpretations  of  Earlier 
Findings  of  this  Report 

3.  Overall  Conclusions  on  the  Effectiveness  of 
the  ATIA  Program  . 


Section  1:    The  American  Management  Association's 
Strategy  for  Change 

Training  Format 

We  defined  the  training  format  as  the  program 
components,  their  contents,  the  people  to  whom  they  were 
administered,  and  periods  in  which  they  were  administered. 


See  Chapter  One  of  this  report  for  a  complete 
explanation  of  the  training  design  of  the  AllA  program. 
This  material  was  adapted  from  the  Yl  evaluation,  Larry 
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The  7iMA  training  package  unites  three  programs: 
the  Management  Course  for  Presidents  (flCP)  ^  the  Top 
flanagement  Briefing  (TMB)  and  the  Educational  Planning 
Process  (EPP) .    The  MCP  was  attended  only  by  the  State 
Superintendent,  while  the  TMB  was  adininistered  to  the 
top  twenty-four  administrators  (twelve  in  the  case  of 
Local  Educational  Agencies)  from  each  State  Education 
Department. 2 

The  third  program,  the  Educational  Planning  Pro- 
cess, reached  "organizational  familif^s":  participants 
included  organizational  superiors  and  subordinates  who 
work  together  regularly.    The  first  group  of  tv/elve  top 
state  agency  administrators  to  attend  the  program  were 
the  State  Superintendent  and  his  immediate  organizational 
subordinates.    The  second  group  was  led  by  the  person 
chiefly  responsible  for  delivery  of  program  services  to 
the  LEA'S.    Groups  from  the  LEA's  were  composed  of 
School  Superintendents  and  eleven  of  their  immediate 
subordinates. 

Almost  all  participants  in  the  EPP  first  attended 
the  Top  Management  Briefing.    Since  the  MCP  was  taken 
only  by  the  State  Superintendent,  wg  will  focus  here  on 
the  TMB  and  EPP  portions  of  the  training  program. 


The  Top  Management  Briefing  and  the  Educational 
Planning  Proces? 

We  believe  that  the  impact  of  training  is  contin- 
gent upon  the  content  conveyed  as  well  as  how  it  is 
conveyed.    Here  we  review  what  has  been  explicated  more 
fully  earlier  concerning  these  aspects  of  the  AMA  train- 
ing program. 

Controls  exercised  upon  the  input  of  the  training 
importantly  influenced  the  type  of  change  that  was  likely 
to  occur.    Analysis  of  the  MCP  and  the  TMP  indicates 
their  considerable  similarity  of  program  design;  thus  we 
will  discuss  only  the  TMB  here. 

The  matrix  developed  as  a  result  of  analyzing  the 
TMB  indicated  that  approximately  80%  of  the  program's 
time  was  controlled  by  the  AMA  lecturer  who  was  sharing 
concepts,  experiences,  and  impressions  with  the  trainees. 


Kirkhart  and  W.  Lynn  Tanner,  "Evaluation  for  Center  for 
Planning  and  Development  of  the  American  'lanagement  Associ- 
ation."   Report  submitted  to  the  American  Management  As- 
sociation and  the  United  States  Office  of  Education, 
Syracuse  University,  October,  1971. 

2 

O  Only  the  State  Superintendent  from  ESED  #2  at- 

ERJC  tended  the  MCP. 
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Approxinately  12.6%  of  training  time  was  dovctGd  to 
general  discussion,  while  7.4%  went  to  small-group 
discussions.    Input  to  the  general  discuBsions  v/as  con- 
trolled by  both  the  leader,  an  A?1A  representative,  and 
the  trainees. 

The  TMPx  strategy  relies  heavily  upon  the  quality 
of  the  information  given  to  the  client,  and  on  the  style 
of  the  lecturer's  presentation. 3  This  general  process 
we  have  called  an  informational  method  of  change. 
Techniques  relying  primarily  upon  information-giving 
are  effective  in  ambiguous  situations,  where  lack  of  - 
information  clearly  prevents  appropriate  performance. 
The  Content  of  the  TMB  centered  on  what  we  have  discussed 
earlier  as  AMA's  view  of  management  problems  in  Education, 
and  on  the  solutions  professional  management  techniques 
can  provide  to  public  agencies. ^ 

The  change  strategy  employed  in  the  second  major 
program,  the  Educational  Plrnning  Process,  differed 
somewhat  from  that  presentc    in  the  TMB.    Unlike  the  T!4B, 
whose  input  was  almost  enti*  ^ly  controlled  by  sources 
external  to  the  trainee,  the  EPP  design  involved  a  mutu- 
ality of  input  control. 

The  boundaries  of  legitimate  discourse  within  the 
EPP  program  were  controlled  by  the  APIA  through  several 
proscribed  steps  in  the  organizational  planning  process 
through  which  the  trainees  are  guided.  During  this  pro- 
cess, the  AMA  attempts  to  build  a  viable  management  team 
through  the  interaction  that  takes  place.    The  ATIA  is 
convinced-  that,  by  dealing  with  problems  that  emerge  in 
the  EPP,  managers  will  modify  their  behavior  and  begin  to 
employ  the  techniques  advocated  by  the  ATm.^ 

While  the  Aim  defined  all  aspects  of  their  planning 
process,  the  input  v/ithin  these  boundaries  emerged  from 
the  client  group  and  the  particular  problems  of  their 
organization.    The  trainer* s  role  v;as  to  clarify  the 
boundaries  of  legitimate  discourse,  keep  the  client  group 
oriented  to  the  problem  of  organizational  planning,  insure 


Cf .  Chapter  One  and  the  Introduction  to  Part  III 
of  this  report. 

4 

Daniel  Katz  and  Robert  L.  Kahn,  The  Social  Psy- 
chology of  Organizations  (New  Yorks  John  Wiley,  1966) , 
p.  393. 

5 

Cf .  Introduction  to  Part  III  of  this  report. 
^Ibid. 
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that  each  step  in  the  planning  process  was  accomplished 
as  £ully  as  possible,  and  attempt  to  maximize  the  extent 
to  which  interpersonal  discourse  remained  a  reason€j5le 
dialogue  and  exchange  of  opinion* 

In  analyzing  the  potential  impact  of  the  AMA 
training  program  the  first  year's  research  team  employed 
a  typology  of  attitude  change  composed  of  compliance, 
identification,  and  internalization. 

Attitude  change  based  on  compliance  involves  doing 
or  saying  what  one  is  expected  to  say  when  a  pewrticular 
situation  is  presented*    Like  change  based  on  compliance, 
attitude  change  based  on  identification  also  requires  an 
external  stimulus  before  the  proper  attit\ide  is  acquired 
by  the  trainee*    Identification  normally  occurs  through 
exposure  to  a  person  whose  social  role  behavior  or 
mannerisms  are  attractive.    The  third  type  of  change  is 
precipitated  by  internalization;  unlike  the  other  two 
attitudinal  changes,  internalization  incorporates  certain 
values  into  the  person's  basic  beliefs  and  does  not 
depend  upon  external  support  for  its  activation. 

Based  on  this  typology,  the  Yl  research  team 
concluded  that  the  TflB — if  it  produced  attitude  change 
at  all — would  do  so  through  compliance  and  identification 
on  the  part  of  the  trainees.^    They  must  identify  and 
accept  the  concepts,  experiences  and  values  of  the  AMA 
lecturers. 

The  primary  processes  of  attitudinal  change  involved 
in  the  EPP  were  also  judged  to  be  compliance  and  identifi- 
cation.   Here,  compliance  was  tied  to  the  fact  that  the 
boundaries  of  discourse  were  defined  by  the  T^'s  con- 
ceptualization of  effective  organizational  planning. 
Identification  processes  were  triggered  by  the  interac- 
tion of  the  members  of  the  client  group.    Attitudes  altered 
by  this  interaction  can  be  expected  to  manifest  themselves 
whenever  the  role  relationships  upon  which  the  identifica- 
tion is  based  are  present. 

Internalized  attitude  change  was  not  considered 
part  of  the  direct  intention  of  the  training  design  by 
the  Yl  research  tearn.^    Their  conclusion  was  reached  on 
the  grounds  that  the  AflA  trainers  did  not  deal  with  the 

frocess  level  of  the  group's  behavior,  and  therefore 
nternalized  change  could  not  occur.    The  emotional 
factors  apparent  in  the  interaction  were  not  dealt  with 
by  the  trainers,  who  intervened  only  to  improve  the 


Kirkhart  and  Tanner,  0£.  cit. ,  p.  128. 
Ibid. 


431 


quality  of  logical  discourse-  and  to  insure  the  continued 
pre-programmed  progress  of  the  grour  through  each  step 
of  the  planning  process.    Since  the  possibility  of 
internalization  is  optimized  v;hen  both  logical  discourse 
and  emotions  are  considered^  the  first  year's  research 
team  concluded  that  internalization  was  unlikely  to  be 
a  major  initiator  of  attitude  change  from  the  AMA  train- 
ing progreim. 


Linking  Program  with  Organizational  Impact 

This  evaluation  (Yl  and  Y2)  rests  on  the  assump- 
tion that  attitudinal  change  is  a  necessary  condition  of 
changing  organizational  behavior,  but  not  a  necessary  and 
sufficient  condition.    The  rjili  training  program  makes  the 
basic  assumption  that  trainees  need  new  knowledge  concern- 
ing management  and  planning  techniques.    Thus,  in  our 
view^  for  the  program  to  be  effective  it  must  change 
attitudes  of  organizational  members  as  well  as  their 
actions  within  their  organizations. 

As  the  Yl  evaluation  concluded  only  a  rhort  time 
after  the  AMA  program  rinded,  the  primary  focus  of  Yl 
analysis  was  measuring  the  impact  of  training  on  attitudes 
related  to  the  original  training  goals  of  the  AMA.    As  the 
planning  documents  had  only  recently  been  developed,  it 
was  considered  inappropriate  to  evalr.te  them  in  Yl. 

The  analysis  of  data  collected  during  the  Yl 
evaluation  indicated  that,  in  general,  the  ATIA  nrogram 
did  not  affect  the  participants'  attitudes  toward  the 
original  AMA  training  goals. 

The  basic  purpose  of  Y2  research  has  been  to 
extend  the  time  persnective  of  the  evaluation  in  order, 
over  time,  to  measure  the  impact  of  AMA  training  on 
organizational  attitudes  and  actions.    The  basic  question 
of  Y3  is,  then:     "To  what  extent  did  the  A!1A  program 
change  the  attitudes  and  actions  of  the  people  in  the 
Experimental  organizations?" 

Our  evaluation  efforts  involved  the  two  State 
Education  Departments  and  the  four  Local  Educational 
Agencies  originally  trained.    As  control  groups,  W3 
added  two  State  Education  Departments  and  two' Local 
Educational  Agencies.^ 

To  link  the  AHA  training  program  with  potential 
organizational  impact  we  have  employed  Likert's  concept 


ERIC 


9 

Sec  Chapter  Two  of  this  report  for  a  complete 
discussion  of  the  Research  Methodology  and  Design 
T^-  employed  in  this  evaluation. 
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of  causal,  intervening,  and  end-result  variables.  We 
1  !!  .  i'""         ^^^'^  training  goals  of  the  AMA~» 

causax  variables.    Their  accomplishAont  is  viewed  here 
Sesf5?2Mn^J°  the  development  of  a  viable  planning  nro- 
cess  within  the  trained  organizations.    Our  definition 
Of  the  intervening  variables  is  based  on  the  AMA's  nrn- 
related  to  leadership  stvle,  decisionmaking, 
and  management  team  relations.  '      L.rL  m^.J^"^^ 

iwn*-«°^ri''-  ^''?  program  affected  these 

organizItiSn  intrinsic  health  of  the 

have                                    °^         end-result  variables  we 
fiave  defined  the  Plans  produced  as  intermediate  output^ 
o^^he  Planning  system.    Final  output  Is  de^lnPd 
gctual,  measured  progress  toward  the  specific  obl^tivps 
stated  in  the  planning  documents: We  regard  each  

IJrSiLSJ'^i^^iJ:  ^"»Per^"^'  ^^^^^  planning  documents 
are  essential  to  the  achievement  of  end-results,  while 
Plans  that  do  not  bring  improved  goal  attainment  have 
■Little  value. 


Section  2;    Summaries  and  Interpretations  of  Earlier 
Findings  ot  this  Report 

This  section  reviews  what  has  been  reported  in 
thl  til  °5tP?^f^*  P^^®  summaries  are  grouped  within 
the  same  divisions  followed  in  Chapters  Four,  Five,  and 

ResiltTar?i.Iel!'''^^'  Intervening  Variables,  and^EnS^ 

4-^^=      "^^^  Charts  presented  in  those  three  chap- 

of  our'^nrf,'!?!!?^^^*^  together  with  an  edited  version 

^fhSil  4         "f.^  conclusions  from  those  charts  as  well  as 
other  information  from  each  chapter.    This  will  re-intro- 

bPt?p^  nnSfJfS''  5°         previous  analysis  so  that  he  can 
oetter  understand  our  summary  conclusions. 


A.    The  Causal  Variables 


The  "causal"  variables  are  indepen- 
dent variables  which  determine  the 
course  of  developments  within  a-\  or- 
ganization and  the  results  achieved 
by  the  organization.    These  causal 
variables  include  only  those  indepen- 
dent variables  v;hich  can  be  altered 
or  changed  by  the  organization  and 
its  management  .  .  . 

-  Rensis  Likert,  The  Human 
Organization  (1967) 


According  to  our  interprotation,  the  causal 
variables  of  this  program  v/ere  the  thirteen  training  goals 
enumerated  by  the  A!1A  as  appropriate  criteria  for  evalua- 
tion.   Each  orgemization  should  have: 

1)  agreed  upon  a  definition  of  the  institu- 
tion's mission; 

2)  esteiblished  continuing  objectives  and 
planning  procedures  for  long-range  achieve- 
ment of  the  institution's  mission; 

3)  identified  resources  and  constraints; 

4)  differentiated  between  where  the  institution 
is  going  and  where  it  wants  to  go; 

5)  modified  previously  established  objectives; 

6)  identified  emd  analyzed  alternative  courses 
of  action; 

7)  determined  priorities; 

8)  made  strategic  action  assignments; 

9)  defined  standards  of  performance  for  key 
adminis  trators ; 

10)  specified  task  completion  dates; 

11)  designed  supplementary  planning  efforts; 

12)  assigned  responsibilities  to  subordinate 
units; 

13)  designed  a  methodology  by  which  future 
performemce  may  be  evaluated  in  relation 
to  the  performances  specified  in  the  plan. 

Based  on  these  criteria,  the  research  team  examined 
work  at  Heunilton  and  immediately  thereafter  in  the  state 
eduqational  departments  and  local  education  agencies. 
We  also  asked  persons  within  those  organizations  how 
ttiey  felt  about  the  theory  behind  their  training  goals, 
as  well  as  how  goals  had  been  put  into  practice.  Our 
attitudinal  inquiries  were  organized  into  several 
categoxifes . 
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1. 
2. 

3. 

4. 
5. 

6. 

7. 


SEP 

Definition  of  the  Mission 
of  the  Institution 

Development  of  Organiza- 
tionnl  Objectives  and 
Priorities 

Mobilization  of  Organiza- 
tional Planning 

Role  of  the  Planning  Unit 

DovGlopment  of  Evaluation 
Techniques 

Top  Managojment  Support  for 
Planning 

The  Credibility  of  the 
Planning  Process 


LEA 

1.  Development  of  Organi- 
zational nission  and 
Objectives 

2.  Mobilization  of  Or- 
ganizational Mission 
and  Objectives 

3.  Top  Management  Sup- 
port for  Planning 

4.  The  Credibility  of 
the  Planning  Process 


This  combination  of  actions  and  attitudes  provided 
^r^nii^^^  ^^^^  assessment  of  the  extent  to  which 

the  AMA  program  affected  the  causal  variables  in  each 
education  agency.    The  follov7ing  charts  summarize  our 
findings,  followed  by  our  interpretations  of  those  charts 
ana  other  data.  . 
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CAUSMi  VARi;*BLES 
STATE  EDUCATION.'^J:.  .'AGENCY  LEVEL 


Summary  of  Findings 
Am  criteria  1-14 
Fall  1970  to  Spring  1972 
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CAUSAL  VARIABLES 
STATE  EDUCATIONAL  AGENCY  LE\^L 
Summary  of  Findings 
iW.  CRITERIA  1-14 
Pall  1970  to  Spring  1972 


WRITTEN  PLANNING  DOCUMENTS 

;  AMOUNT  OF  PROGRESS  

—  Criteria  ninimum     Moderate  Maximum 

1.  Agreed  upon  a  definition  of 

the  institution's  mission  E1/E2 

2.  Established  continuing  ob-^ 
jectives  and  planning  pro- 
cedures for  long-range 
achievement  of  the  insti- 
tution's mission  E1,E2 

3.  Identified  resources  and 

constraints  E1,E2 

4.  Differentiated  between 
where  the  institution  is 
going  and  where  it  wants 

5.  Modified  previously  estab- 
lished objectives  E2 

6.  Identified  and  analyzed 
alternative  courses  of 

action  E1,E2 

7.  Determined  priorities  e1  E2 

8.  flade  strategic  action 

assignments  f1  e2 

9.  Defined  standards  of  per- 
formance for  key  adminis- 
trators PX  JJ2 

10.  Specified  task  completion 

dates  T?i  22 

11.  Designed  supplementary 

planning  efforts  E1,E2 

12.  Assigned  responsibilities 

to  subordinate  units  E1,E2 

13.  Designed  a  methodology  by 
which  future  performance 
may  be  evaluated  in  rela- 
tion to  the  performances 
specified  in  the  plan  E1,F2 

14.  Produced  and  are  implementing 

a  long-range  strategic  plan  E1,F2 
Total  Number  of  Possible  Effects :28   5  16  7 
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ATTITUDINAL  DATA  SUriMARY 

. Causal  Variables  -  State 
Education  Agencies 

Fall,  1970     -     Spring,  1972 


ITIPACT  OF  TRAINING 


Positive     JTo  Negative 
Item  Typo  of  Data  Effect    Effect  Effect 

DEFINITION  OF  THE  MISSION  OF  THE  ORGA^TIZATION 
CONTENT  DATA  DRAWN  FROM  INTERVIEWS 

1  Definition  of  the  Institu- 
tion's Mission  El  E2 

2  Sense  of  SED  Mission  E1,E2 

Feelings  about  the  direction 
the  organization  is  moving  E1,E2 


QUESTIONNAIRE  ITEMS 

4    The  kinds  of  things  I  an  do- 
ing will  make  a  long-term 

contribution  to  education.  El  E2 

DEVELOP^IENT  OF  ORGANIZATIONAL  OBJECTIVES  &  PRIORITIES 
CONTENT  DATA  DRAWN  FROM  INTERVIEWS 

1  Modify  previously  established 

objectives  El  E2 

2  Identify  and  analyze  alterna- 
tive courses  of  action  E1,E2 

3  Determine  priorities  E1/E2 

4  Do  you  feel  that  the  objec- 
tives developed  as  a  result 
of  A*1A  training  reflect  the 
most  serious  and  pressing 

needs  of  state  education?  E1/E2 

QUESTIONNAIRE  ITEMS 

5  The  goals  of  this  organiza- 
tion are  articulated.  El,E2 

6  Our  goals  are  realistic  and 
attaineible  with  our  best 

efforts.  E1,E2 

7  The  top-priority  objectives 
of  state  education  are  clear 

to  me.  E1,E2 
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IMPACT  OF  TR?\INING 


Item 
8 


4 
5 


8 


10 


1 
2 


Typo  of  Data 

I  feol  that  the  objectives 
developed  during  AMA  training 
reflect  the  most  serious  and 
pressing  needs  of  state 
education. 

As  I  see  it,  the  operational 
priorities  of  the  objectives 
developed  during  Mik  train- 
ing are  clear. 


Positive 
Effect 


No 
Effect 


E1,E2 


Negative 
Effect 


El,E2 


MOBILIZATION  OF  OPvGAinZATIONAL  PIJ^TNING 

CONTENT  DATA  DRAWN  FROM  INTERVIEWS 

Define  standards  of  perfor- 
mance for  key  administrators  E1,E2 

Specify  task  completion  dates 

and  action  assignments  E1,E2 

Assign  responsibilities  to 
subordinate  units  e2 

Need  for  Performance  Standards 

Performance  Standard8--Extent 
of  Use 

Existence  of  Performance 
Reviews 

QUESTIONNAIRE  ITEMS 

My  organization's  policy 
statements  are  clear. 

My  organization's  performance 
standards  are  understood.  El,r2 

Good  ways  are  used  to  let  me 
know  how  I  can  improve  my 

performance.  El,E2 

I  understand  what  results 
must  be  produced  to  achieve 
the  stated  objectives  of 

this  organization  e2 

ROLE  OF  THE  PLANNING  UNIT 

CONTENT  DATA  DRAt'TN  FROM  INTERVIEWS 

Awareness  of  need  to  evalu- 
ate our  programs  El,E2 

Availaible  to  answer  planning 

questions  E1,E2 


El 

INSUFFICIENT 
DATA  (T4  ONLY) 
ON  WHICH  TO 
BASE  A  RESPONSE 


E1,E2 


FA 
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IffPACT  OP  TRAINING 


Positive 
Effect 


4 
5 


Item  Type  of  Data 

3      Writing  guidelines  for  plan 
development 

Reviewing  and  refining  plans 

Provides  leadership  in  the 
implementation  of  planning 

Provides  in-service  training 
in  planning 

QUESTIONNAIRE  ITEMS 

The  planning  unit  has  been 
helpful  to  me. 


DEVELOPMENT  OF  EVALUATION  TECHNIQUES 

CONTENT  DATA  DRAWN  FROM  INTERVIEWS 

Designed  a  methodology  by 
which  future  performance  may 
be  evaluated  in  relation  to 
the  performance  specified 
in  the  plan 

Informal  Feedback 


2 
3 
4 
5 
6 

7 
8 


1 
2 


No 
Effect 


El 
tl,E2 

E1,E2 

E1,E2 


E1,E2 


Negative 
Effect 


E2 


E1,E2 
INSUFFICIENT 
DATA  (T4  ONLY) 
ON  I'JHICH  TO 
BASE  A 
RESPONSE 


Performance  Reviews 
Questionnaires 
Task  Completion  Inventories 
Unobtrusive  Measures 

QUESTIONNAIRE  ITEMS 

I  have  good  ways  for  knowing 

how  good  our  results  arc.  E2 

My  organization  has  reliable 
ways  for  knowing  how  well  it 
is  attaining  its  objectives.  E1,E2 

I  think  that  the  objectives 
developed  during  .AMA  train- 
ing are  clearly  stated  with 
respect  to  results  expected.  E1,E2 

TOP  MANAGEMENT  SUPPORT  FOR  PLANNING 

CONTENT  DATA  DRAWN  FROM  INTERVIEWS 
Adequate  resources  (money 

and  information)  E1,E2 

Control  System  expressed 
through  decision  making 

process  El,E2 


El 
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IMPACT  OF  TRAINIim 


Item 


8 


1 
2 


8 


Type  of  Data 

QUESTIONMAIRE  ITEMS 

My  manager  makes  it  clear  ho 
is  committed  to  the  success 
of  our  projects. 

Mv  manager  has  expressed  the 
belief  that  the  Ally's  train- 
ing program  has  been  helpful. 

My  manager  understands  plan- 
ning theory  and  is  able  to 
put  it  into  practice. 

I  bolicvo  my  organization 
gives  me  adequate  training 
to  do  my  work  effectively. 

I  feel  good  eJuout  my  meuia- 
ger's  ability  to  plan. 

My  manager  provides  me  with 
adequate  support  to  perform 
my  job. 


Positive 
Effect 


No 
Effect 


Negative 
Effect 


El 


El 


E1,E2 

E1,E2 

E1,E2 
E1,E2 


El 


E2 


THE  CREDIBILITY  OF  THE  PLANNING  PROCESS 

CONTENT  DATA  DRAWN  FROM  INTERVIEWS  . 

Establish  credibility  of 
planning 


Role  of  Planning: 
integral 

Role  of  Planning: 
is  needed 


how 


how  much 


E1,E2 


Role  of  Planning:    emergence  E1,E2 

QUESTIONNAIRE  ITEMS 

As  I  see  it,  planning  is  an 
integral  part  of  running 
the  state's  schools. 

As  I  see  it,  persons  in 
this  organization  put  a 
lot  of  effort  into  planning. 

My  capability  to  nlan  effec- 
tively will  positively  affect 
my  future  caroer  in  this 
organization.  El,E2 

The  activities  relating  to 
planning  are  having  an  effect 
on  the  policy  of  this  organi- 
zation. 

Total  Number  of  Possible  Effects: 84  12 


El 


E1,E2 


E2 


E1,E2 


E1,E2 


El 

54 


E2 
18 
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CAUSAL  VARIABLFS 
LOCAL  EDUCATIONAL  AGEliCY  LEVEL 


Summary  of  Findings 
mr.  Criteria  1-14 
Fall  1970  to  SprincT  1972 
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CAUSAL  VARIABLES 

LOCAL  EDUCATIOIIAL  AGENCY  LEVEL 

Suimnary  of  Piridinas 
AIIA  CRITERIA  1-14 
Fall  1970  to  Spring  1972 


WRITTEN  PLANNING  DOCUMENTS 


No.  Criteria 

1.  Agreed  upon  a  definition  of 
the  institution's  mission 

2.  Established  continuing  objec- 
tives and  planning  procedures 
for  long-range  achievement 

of  the  institution's  mission 

3.  Identified  resources  and 
constraints 

4.  Differentiated  between  where 
the  institution  is  going 
and  whore  it  wants  to  go 

5.  Modified  previously  estcib- 
lished  objectives 

6.  Identified  and  analyzed  al- 
ternative courses  of  action 

7.  Determined  priorities 

8.  Made  strategic  action  as- 
signments 

0.     Defined  standards  of  perfor- 
mance for  key  administrators 

10.  Specified  task  completion 
dates 

11.  Designed  supplementary 
planning  efforts 

12.  Assigned  responsibilities 
to  subordinate  units 

13.  Designed  a  methodology  by 
V7hich  future  performance  may 
be  evaluated  in  relation  to 
the  performances  specified 
in  the  plan 

14.  Produced  and  are  implement- 
ing a  long-range  strategic 
plan 


AflOUNT  OF  PROGPESS 
Minimum    ?loderate  Maximum 

EL1,EL2 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 

EL1,EL2, 
KL3  EL4 

EL1,EL2, 
EL3 , EL4 

EL3        ■  EL1,EL4 
EL1,EL2, 
EL3,EL4 


EL2 


EL3 

EL1,EL2, 
EL3,EL4 


EL3 


EL2 , EL4  ELI 


EL1,EL3  EL2,EL4 

EL1,EL2, 
FL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3 , EL4 


EL1,EL2, 

EL  4 


Total  Number  of  Possible  Effects: 55  16 


27 


12 
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su^c^;^^Ry  of  attitudinal  findings 

CAUSJxL  VARIABLES  - 
LOCxAL  EDUCATION  AGENCY  LEVEL 
Fall  1970      ~      Spring  ir/72 


IflPACT  OF  TRAINING 


Posi  t  ive         No  Negative 
Item  Type  of  Data  Effect       Effect  Effect 

DEVELOPMENT  OF  ORGMtlZATIONAL  MISSION  &  OBJECTIVES 

QUESTIONNAIRE  ITEMS 

1  The  goals  of  this  organiza-  EL2,EL3, 
tion  are  articulated.  ELI  EL4 

2  Our  goals  are  realistic  and 
attainable  with  our  best 

efforts .  ELI ,EL2      EL3 , EL4 

3  The  top  priority  objectives 

for  state  education  are  FLl,EL2r 
clear  to  me.  EL3,EL4 

4  I  feel  that  the  objectives 
developed  during  At?A  train- 
ing reflect  the  most  seri-* 

ous  and  pressing  needs  of  EL1,EL2, 

state  education.  EL3,EL4 

5  As  I  see  it,  the  organiza- 
tional priorities  of  the 

objectives  developed  dur-  ELl,EL2, 
ing  ATiA  training  nro  clear.  EL3,EL4 

6  The  kinds  of  things  I  am 

doing  will  make  a  long-term  ELl,EL2r 
contribution  to  education.  EL3,EL4 

7  As  I  see  it,  my  organization 

is  moving  in  the  right  EL1,EL2, 
direction.  EL3 , EL4 

MOBILIZATION  OF  ORGANIZATIONAL  PLANNING 
QUESTIONNAIRE  ITEMS 

1  My  organization's  policy  EL1,EL2, 
statements  are  clear.  EL3,EL4 

2  My  organization's  perfor-  EL2,EL3, 
memce  standards  are  clear.  ELI  EL4 

3  Good  ways  are  used  to  let  me 
know  how  I  can  improve  my 

performance.  EL1,EL2  EL3,EL4 
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IMPACT 
itive 


OP  TRAINING 


Pos 
Effect 


Item  Typo  of  Data 

4  I  understand  what  results 
must  be  produced  to  achieve 
the  stated  objectives  of 
this  organization. 

5  The  planning  unit  has  been 
helpful  to  me. 

6  I  have  good  ways  for  knowing 
how  good  our  results  are. 

7  My  organization  has  reliable 
ways  for  knowing  ha^  well 

it  is  achieving  its  objectives, 

8  I  think  that  the  objectives 
developed  during  l\m  training 
arc  clearly  stated  with  re- 
gard to  results  expected. 


TOP  MANAGEMENT  SUPPORT  FOR  PLANNING 
QUESTIONNAIRE  ITEMS 

1  My  manager  makes  it  clear  he 
is  committed  to  the  success 
of  our  projects. 

2  lly  manager  has  expressed  the 
belief  that  the  AMA's  train- 
ing program  has  been  helpful.  EL3,EL4 

3  My  manager  understands  plan- 
ning theory  and  is  able  to 
put  it  into  practice. 

4  I  believe  my  organization 
gives  me  adequate  training 
to  do  my  work  effectively. 

5  I  feel  good  aibout  my  mana- 
ger's ability  to  plan. 

6  My  manager  provides  me  with 
adequate  support  to  perform 
my  job. 

CREDIBILITY  OF  THE  PLANNING  PROCESS 
QUESTIONNAIRE  ITEMS 

1  As  I  see  it,  planning  is  an 
integral  part  of  running 
the  state's  schools. 

2  As  I  see  it,  persons  in  this 
organization  put  a  lot  of 
effort  into  planning. 


No 
Effect 


Negative 
Effect 


EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2 


ELI, EL 2  EL3,EL4 

ELl,EL2, 
EL3,EL4 

EL1,FL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3 


EL4 
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IMPACT  OF  TRAiniNG  

Positive  iTo  Nogative 

-t^^            Typo  of  Data                     Flffect  Effect  Effect 

3  My  capability  to  plan 
effectively  v^ill  positively 

affect  my  future  career  EL1,EL2, 

in  this  organization.  EL3,EL^. 

4  The  activities  relating  to 
planning  are  having  an 

effect  on  the  policy  of  EL1,EL2, 

this  organization  EL3,EL4 

Total  number  of  Possible  Effectsrioo     7  87  6 


Interpretation 

An  analysis  of  actual  results  on  the  first  thirteen 
criteria  indicates  that,  with  few  exceptions,  the  SED's 
and  LEjP.'s  had  made  moderate  progress  on  all  of  them.  The 
research  team  believed  that  most  of  the  action  taken  on 
most  of  t  2  criteria  demonstrated  that  the  organizations 
had  done  more  than  just  mention  the  criterion  (ninimum 
progress);  but  they  had  not  actually  fully  developed  it 
(maximum  progress).    WIA  training  definitely  helped  the 
educational  agencies  to  devote  considerable  time  and 
energy  to  planning  and  to  produce  a  wide  range  of  plan- 
ning documents.    At  least  on  paper,  the  germs  of  a 
satisfactory  planning  process  were  present. 

However,  a  survey  of  attitudinal  findings  related 
to  these  planned  actions  reveals  that,  to  a  large  extent, 
the  planning  documents  had  not  changed  hew  people  felt 
about  organizational  planning.    A  pattern  of  no  effects 
emerged  when  we  compared  attitudes"  expressed  55'fore  train- 
ing with  attitudes  expressed  after  training  and  after  the  , 
actions  had  been  taken. 

Training  had  not  altered  participants'  impressions 
of  their  ongoing  planning  process.    By  and  large,  the 
Experimental  agencies  thought  they  were  doing  about  the 
same  quantity  and  quality  of  planning  before  training 
began  ard  after  training  ended.    While  some  effects 
might  have  been  observed  immediately  after  training, 
many  disappeared  within  a  year.    The  7JLA  program  did  very 
little  to  change  permanently,  for  example,  attitudes  on 
the  sense  of  organizational  mission,  objectives,  perfor- 
mance standards,  and  job  descriptions. 

?s  a  result,  if  the  organization  had  considered 
itself  effective  by  the  standards  of  certain  training 
goals  before  training,  they  had  not  changed  their 
opinions  after  training.    On  the  other  han^,  if  an 
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organization  had  thought  liltJe  of  it53  planning  opera- 
tions prior  to  7\j;r.  training,  the  program  did  not  sub- 
stantially improve  these  opinions i 

Thus,  in  terns  of  how  persons  in  the  SED's  and 
lea's  felt  about  their  planning  process,  a  strong  pro- 
cess remained  strong  and  a  woak  process  remained  v/eak. 
Despite  the  spate  of  planning  documents  produced  as  a 
consequence  of  2\?m  training,  opinions  about  the  plan- 
ning process  in  the  organization  itself  remain  unchanged. 

This  raises  the  significant  issue  of  how  viable, 
in  fact,  these  documents  were.    If  they  were  usable 
action  papers,  we  would  assume  that  the  plans  would 
have  affected  the  attitudes  of  those  who  developed  and 
worked  with  them.    That  the  plans  had  virtually "no 
effect  suggests  that  much  ot  v'hat  was  done  during  and 
after  training  was  seen  as  meaningless  paper  production, 
unrelated  to  the  actual  process  by  which  plans  were,  or 
were  not,  developed  inside  the  agencies. 


B.    The  Intervening  Variables 

The  "intervening"  variables  reflect 
the  internal  state  and  health  of  the 
organization,  e.g.,  the  loyalties, 
attitudes,  motivations #  performance 
goals,  and  perceptions  of  all  members 
and  their  collective  capacity  for 
effective  interaction,  communication, 
and  decision  making. 

Rensis  Likert,  The  Human 
Organization  (1967) 


The  AHA  has  specific  goals  concerning  these  inter- 
ven.nc  variables  that  are  crucial  to  an  evaluation. 
These  variables         the  essential  ingredients  of  the 
organizatioi^al  environment  that  can  make  or  mar  the 
team  planning  process. 

'2hiB  process  defends  on  the  development  of  strong 
leadership  to  guide  the  organization;  decentralization 
of  decisionmaking  so  that  planning  decisions  can  be 
made,  as  much  as  poscible,  by  those  nearest  the  level 
at  which  the  decision  will  be  implemented;  and  the 
creation  of  teaitis  of  managers  who  cooperate  with  each 
other  to  achieve  jointly  the  educational  objectives 
specified  in  the  plan. 


Without  a  suitable  internal  environment,  even 
complete  development  and  internalization  of  the  thirteen 
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AMA  training  goals  (causal  variables)  cannot  be  trans- 
lated into  viable  organizational  plans  and  actual 
progress.    To  use  an  agricultural  analogy,  intervenina 
variables  constitute  the  ground     in  which  the  seeds  of 
the  causal  variables  are  Planted,  and  from  v;hich  will 
hopefully  grow  mature  plans  and  nrogress  toward  plan 
objectives. 

To  evaluate  those  variables,  we  developed  three 
categories.    Each  contained  interview  and  questionnaire 
items,  analysis  of  which  gave  the  research  team  a  com- 
prehensive view  of  the  ef fectiven'-:ss  of  the  training 
program  in  that  category.    The  several  categories, 
taken  together,  were  the  basis  of  overall  conclusions  as 
to  training's  effect  upon  the  internal  workings  of  the 
organizations.    These  categories  were: 

A.  Leadership  Climate 

B.  Decisionmaking 

C.  Management  Team  Relations 

Data  summary  charts  presenting  the  results  of  our 
interviews  and  questionnaires  are  renrinted  from  Chapter 
Five  belov;.    Following  the  charts  is  an  edited  version 
of  our  original  interpretation  of  the  data. 
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INTERVENING  VARIABLES 
STATE  EDUCATIONAL  AGENCY  LEVEL 

Summary  of  Findincrs 
Pall  1970  to  Spring  1972 
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.''.TTITUDINAL  DATA  SUfmHY 


Intervening  Variables  -  state 
Educational  Agency  Level 

Fall  1070    ~  Spring  1972 


Item  Type  of  Data 

I.EADERSHIP  CLIMATE 

QUESTIONNAIRE  ITEflS 

1      Based  on  information  I  have 
received  from  my  boss,  I 
knov/  if  I  am  measuring  up 
in  ray  job. 

My  manager  encourages  and 
supports  innovation. 

Higher  management's  reac- 
tions to  the  problems  which 
reach  them  are  fair. 

My  manager  knows  and  under- 
stands the  problems  I  face. 

My  manager  recognizes  when 
a  problem  is  developing  and 
does  something  constructive 
about  it. 

My  manager  shows  confidence 
and  trust  in  me. 

DECISIONMAKING 


2 
3 


IMPACT  OF  TRAINING 

Positive     No  Negative 
Effect    rffect  Effect 


2 
3 


El 
Fl 

E2 

El 

Ei 

El 


CONTENT  DATA  DRAWN  FROM  INTERVIEWS 

Involvement  in  Decision- 
making in  the  State  Depart- 
ment 

Quality  of  Decisionmaking 

in  the  State  Department  El,E2 

Influence  of  Planning  on 
decisionmaking  provess 

QUESTIONNAIRE  ITEMS 

The  people  I  work  with  par- 
ticipate appropriately  in 
setting  the  goals  of  our  El 
work. 


E2 
E2 


El 


E2 

E2 
E2 


E1,E2 


E1,E2 


E2 
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I torn  Typo  of  Data 

^ 

5  I  am  appropriately  involved 

in  decisions  affecting  my  work.  Fl,F2 

6  I  can  influence  the  goals, 
methods,  and  activities  of 
my  organization. 

flANAGEMENT  TEAIi  RELATIONS 

CONTENT  DATA  DRAWI^  FROM  INTERVIEWS 
Promote  cooperative  teamwork 
Amount  of  cooperative  teamwork 


rtPACT  OF  TPJ'INIMG 

Positive    Ho  Negative 
Fffcct  Effect  Fffect 


1 
2 

3 
4 
5 
6 


QUESTIONIIAIRE  ITEJtS 

My  work  group  works  hard  to 
achieve  its  goals. 

My  work  group  understands 
V7hat  we  are  trying  to  achieve. 

I  feel  my  group  works  well 
together. 

I  really  feel  my  immediate 
work  group  is  getting  things 
done. 

When  differences  arise  in  my 
work  group,  we  have  good  ways 
for  settling  them  ourselves. 


El 


E1,E2 


E2 


El 


E2 


E1,E2 


El 
E2 


El 
El 


Total  Number  of  Possible  Effects: 38  17 


El 
12 


E2 


E2 


E2 
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INTEn^/ENIIIG  VARIABLES 
LOCAL  EDUCATIOr.AL  AGENCY  LEVEL 

Suuiraary  of  Pindingp 

Pall  1970  to  Snrino  1972 
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ATTITUDINAL  DATA  SUIIMARY 

Intervening  Variables  -  Local 
Educational  Agency  Level 

Fall  1970    -   Spring  1972 


Item  Type  of  Data 

LE/yjERSHIP  CLIIIATE 

QUESTIOtJNAIRE  ITEIIS 

1  Based  on  information  I  have 
received  from  my  boss,  I 
know  if  I  am  measuring  up 
in  my  job. 

2  My  manager  encourages  and 
supports  innovation. 

3  Higher  management's  reactions 
to  the  problems  which  reach 
them  are  fair. 

4  riy  manager  knows  and  under- 
stands the  problems  I  face. 

5  Ky  manager  recognizes  when  a 
problem  is  developing  and  does 
something  constructive  about  it 

6  My  manager  shows  confidence 
and  trust  in  me. 

DECISIONMAKING 

QUESTIONNAIRE  ITEMS 

1  The  people  I  work  vrith  parti- 
cipate appropriately  in  set- 
ting the  goals  of  our  work. 

2  I  am  appropriately  involved  in 
decisions  affecting  my  work. 

3  I  can  influence  the  goals, 
method:;,  and  activities  of  my 
organization. 

MAN.^.GEMENT  TEA.M  RELATIONS 
QUESTIONNAIRE  ITEIIS 

1  My  work  group  works  hard  to 
achieve  its  goals. 

2  My  work  group  understands  what 
we  are  trying  to  achieve. 


I?^FACT  OF  TRAINING 

Positive     No  Negative 
Effect    Effect  Effect 


EL1,EL2 
EL3,EL4 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 

ELI 
EL  2 

FL1,EL2, 
EL3,EL4 


FL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL3 
EL4 
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IMFr.CT  OF  TRAINING 


Item  Type  of  Data 

3  I  feel  my  group  works  well 
together. 

4  I  really  feel  my  immediate 
work  group  is  getting 
things  done. 

5  When  differences  arise  in 
my  work  group  we  have  good 
ways  for  settling  them 
ourselves. 


Positive 
Effect 


?To 
Effect 

EL1,EL2, 
EL3,EL4 

EL1,EL2, 
EL3,EL4 


EL1,EL2, 
EL3,EL4 


Negative 
Effect 


Total  Number  of  Possible  Effects; 56  0 
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Interpretation 

The  SED's  and  LEA's  exhibited  different  training 
effects.    ?anong  the  SED's,  the  quality  of  El's  interven- 
ing variables  were  often  improved  by  training,  while  in 
E2  the  same  variables  were  frequently  damaged  by  partici- 
pation in  the  AMA  program.    The  LEA's  assumed  a  middle 
position  both  in  reference  to  the  two  Experimental  States 
and  on  the  data  summary  chart;  no  effects  of  training  were 
the  rule,  not  the  exception,  in  these  org€uiizations. " 

These  changes  did  not  occui  across-the-board,  but 
reflected  the  general  trend  of  the  data  for  each  agency. 
Not  all  items  for  El  revealed  positive  effects;  nor  did 
every  indicator  for  E2  show  negative  effects. 

What  do  these  results  mean  for  the  plemning  pro- 
cess?   As  sufficient  but  not  necessary  conditions  for 
effective  planning,  the  inteanrening  variables  are  crucial 
to  the  A?1A  program.    A  healthy  internal  environment  does 
not  guarantee  effective  planning,  but  effective  planning 
cannot  genuinely  occur  without  it.    Fertile  ground  does 
not  always  yield  good  crops,  but  good  crops  cannot  grow 
in  infertile  soil. 

The  data  shows  that  State  Educational  Agency  El's 
internal  environment,  to  a  certain  extent,  was  more 
favorable  to  team  planning  after  training  than  they  had 
been  before;  E2'3  was  less  favorabl3.    The  LEAs*  environment 
was  unchanged .    Because  of  the  intervening  variables' 
potential  contribution  to  organizational  planning,  we  sur-- 
mise  that  El's  chances  for  developing  a  viable  planning 
process  were  enhanced;  E2's  vrere  diminished;  and  the 
lea's  wore  uneffected  by  the  AJIA  program. 
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Our  judgment  is  that  AIlA's  objective  of  improving 
thG  loadorship  climate,  decisionmaking  process,  and 
management  team  relations  of  these  six  educational 
agencies  had  a  16%  success  rate. 

Because  of  the  positive  effects  that  were  measured, 
the  training  had  its  intended  impact,  on  many  items,  in 
El.    Yet  the  program  v/as  largely  unsuccessful  in  promot- 
ing favorable  internal  variables  in  the  other  Experimental 
State  or  in  the  LEA's.    The  AMA  failed  in  E2  because  of 
negative  training  effects  and  in  the  four  school  districts 
which  registered  no  effects  at  all. 


C.    The  End-Result  Variables 


The  end-result  variables  are  the 
dependent  variables  which  reflect 
the  achievements  of  the  organiza- 
tion. .  . 

-  Rensis  Likert,  The  Human 
Organization  {1967) 


Analysis  of  the  end-result  variables  answered  the 
question,  "What  happened  as  a  result  of  the  ATIA  program?" 
The  causal  and  intervening  variables  laid  groundwork  for 
the  development  of  ongoing  divisional  plans  and  actual 
progress  toward  educational  objectives  therein.  Comple- 
tion of  the  thirteen  A?IA  training  goals  and  the  develop- 
ment of  a  favorable  internal  environment  were  essential 
for  genuine  planning  and  goal  attainment. 

As  was  necessary  in  our  consideration  of  previous 
variables,  actions  and  attitudes  became  the  twin  centers 
of  attention  when  end-results  arc  assessed.    Even  if 
plans  and  progress  are  being  made,  if  individuals  believe 
that  their  plans  remain  undeveloped  and  their  progress 
is  nonexistent,  the  process  will  collapse  from  lack  of 
support.    If  organization  members  believe  that  plans  are 
inoperable  and  progress  impossible,  chances  are' that 
they  are  right.    Actions  and  attitudes  more  often  agree 
than  disagree,  even  if  we  do  not  consider  the  social- 
psychological  reality  of  the  self-fulfilling  prophecy. 

Our  examination  of  end-result  variables  took  two 
forms,  the  first  an  analysis  of  SEP  priority  area  plans 
and  LEA  general  plans.    In  both  cases,  widely  accepted 
criteria  of  what  constitutes  an  operational  plan  v/ere 
used.    Secondly,  we  asked  organization  members  what  they 
thought  about  their  plans  and  progress.    Prom  these  sources, 
we  drew  our  interpretations  and  conclusions. 

Raymond  E.  Klawuhn  and  Alexander  J.  Basso,  "Final 
Report:    Adapting  and  Testing  Business  Management  Develop- 
FRir"  ^^^^  Programs  for  Educational  Administrators"  (mimeo) , 

Jan.  1972,  p.  29. 
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END  PJ2SULT  VARIABLES 
STATE  EDUCATI0N7.L  AGE^TCY  LEVEL 

Summary  of  Findings 
Fall  1970  to  Spring  1572 
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ANALYSIS  OF  STATE  EDUCATIONAL  AGEITCY 
PRIORITY  AREA  PLANS 

End  Result  Varieibles 
Spring  1972 

ESED  #1 


Career 

Educa-  Read- 
tion  ing 


ESED  f-2 


Early  Human 
Child-    Re la-  Read- 
hood      tions  ing 


CRITERIA 


ANALYSIS 


1.  Title 


2. 


Identifi- 
cation 


3.  Needs  As- 
sessment 

4 .  Recommen- 
dations 

5.  Expected 
Results 

6 .  Time 
Frames 

7.  Responsi- 
bility 

8.  Resource 
Require- 
ments 

9 .  Evalua- 
tion 

Strategies 
10.  Rationale 


Yes 

No 

Yes 

No 

Yes 

Yes 

No 

No 

No 

NO 


Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

Yes 

No 


Yes 

No 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 
No 


Yes 
No 

Y2S 

Yes 
Yes 
Yes 
No 

^to 

No 
No 


Yes 

Yes 

No 

Yes 

Yes 

Yes 

Yes 

Yes 

No 
No 


Yes=4  Yes=6 
No  «6      No  =4 


Yes=5  Yes=5  Yes=7 
No  =5      No  =5      No  =3 
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ATTITUDINAL  DAT7i  SUMIl'VRY 

End  Result  Variables 
STATE  EDUCATIONAL  AGENCIES 

Fall  1970  -  Spring  1972 


II H? ACT  0?  TPvAiriNG 


I  torn 


Tyne  of  Data 


Positive       No  Neaative 
Effect      Effect  Effect 


1 
2 


DEVELOPi'lENT  OF  A  LONG  RT^JGE  STRATEGIC  PLAN 
CONTENT  DATA  DrJkfflJ  FROM  INTERVIEWS 

Produce  and  implement  n  long- 
range  strategic  plan 


QUESTIONNAIRE  ITEMS 

My  organization's  overall  plan 
is  operable. 

PROGRESS  TOWARD  GOAL  ACHIEVEMENT 

CONTENT  DATA  DRAWN  F^OM  INTERVIEWS 

Number  of  objectives  toward 
which  progress  has  been  made 

Level  of  Progress  tov;ard 
those  objectives 

QUESTIONNAIRE  ITEMS 

As  I  see  it,  my  organization 
has  made  progress  in  attain- 
ing its  objectives. 


Total  Number  of  Possible  Effects-.:  10  0 


El 


E1,E2 


E1,E2 
E1,E2 


F1,E2 

9 


E2 


ERIC 
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END  PESULT  VARIABLES 
LOCAL  EDUCATIONWti  AGENCY  LEVEL 

Summary  of  Findings 
Fall  1970  to  Spring  1972 


/ANALYSIS  OF  LOCAL  EDUCATIONAL  AGENCY  PLANS 
Enr*  result  Variables 
Spring  1972 


ELErifl 

ELEA^t  2 

ELEA#  3 

CRITERIA 

A  N 

A  L  Y  S  I  S 

1 . 

Title 

Yes 

Ygs 

Yes 

2. 

Idenrif ication 

Ilo 

No 

No 

3. 

Needs  Assessment 

No 

No 

No 

4. 

recommendations 

Yes 

Yes 

Yes 

5. 

Expected  Results 

Yos 

Yes 

No 

6. 

Time  Frames 

Yes 

Yes 

No 

7. 

Responsibility 

No 

No 

No 

8. 

Resource  Requirements 

No 

Yes 

Wo 

9. 

Evaluation  Strategy 

No 

No 

No 

10. 

Rationale 

No 

No 

No 

Yes=4 

Yes=5 

Yes=2 

No  =6 

No  =5 

No  =8 

463 


ATTITUDINAL  DATA  SUMMAPsY 

End  Result  Variables 
LOCAL  EDUCATIONAL  AGENCY  LEVEL 
Fall  1970     -     Spring  1972 

IMPACT  OF  TRAINING 


Positive      No  Negative 
Item  Type  of  Data  .  Effect^  Effect  Effect 

DEVELOPMENT  OF  A  LONG  RANGE  STRATEGIC  PLAN 

QUESTIONNAIRE  ITEMS 

1     My  organization's  overall  EL3 
plan  is  operable.  EL1^EL2  EL4 

PROGRESS  TOWARD  GOAL  ACHIEVEMENT 

QUESTIONNAIRE  ITEMS 

1     As  I  see  it^  my  organiza- 
tion has  made  progress  in  EL1^EL2 
attaining  its  objectives.  EL3^EL4 


Total  Number  of  Possible  Effects: 8 


2 


Interpretation 

We  found  plans  and  progress  not  completely 
acceptable.    Concerning  the  plans ^  we  determined  that 
their  objectives  and  strategies  were  well  developed  anc 
contained  practical  guidelines  to  assess  their  attain- 
ment (percentage  achievement  levels,  cli  nt  groups, 
and  deadline  dates).    However,  several  re^Jfuired  elements 
for  an  operational  plan  were  often  missing. 

Systematic  analyses  of  the  needs  that  underlay 
action  objective  or  strategy  were  not  integral  parts 
of  the  plans  s^en  by  the  research  team.    Nor  were  the 
personnel  and  money  required  to  reach  an  objective 
outlined  in  relation  to  that  objective.  Responsibility 
for  reaching  specific  objectives  or  strategies  was 
frequently  left  obscure.    While  the  plans  concisely 
stated  where  the  agencies  wished  to  go,  they  contained 
few  evaluation  methods  to  tell  how  fast  the  organiza- 
tions were  moving  or  whethor  they  had  arrived. 

This  was  reflected  in  the  attitudes  of  members 
of  the  SED*s  and  LEA's.    One  of  the  reasons  given  by 


the  AMA  for  applying  business  managemont  rlcvclopment 
programs  to  education  v;^.s  that: 

Realistic,  measurable  objectives 
appear  to  be  nonexistent  in  many 
educational  agencies,  and  when 
defined  they  are  frequently  vague, 
not  explicit,  and  unmeasurable. 

If  this  was  the  situation  prior  to  AMA  training,  then, 
according  to  program  particir^ants ,  not  much  has  ^ '^anged 
as  a  result  of  -^-raining.    No  significant  changes  were 
recorded  in  views  of  the  operability  of  agency  plans  or 
in  aggregate  awareness  of  progress  toward  objectives. 

SED's  and  LEA's  that  considered  their  plans  effec- 
tive and  their  progress  steady  before  training  did  not 
substantially  change  their  *^iews  over  time.  Conversely, 
if  plans  were  thought  unrealistic  or  unmeasurable  before 
AMA  training,  tbey  were  seen  the  same  way  after  training. 
One  of  the  reasons  why  opinions  did  not  improve  (and  in 
some  cases  they  declined),  was  because  some  plans  were 
indeed  unrealistic  and  unmeasurable.    The  absence  of 
needs  statements,  cost  estimates,  rpsponsibility  assign- 
ments, and  evaluative  methods  made  them  less  than  realis- 
tic.   The  ATIA  put  an  army  of  objectives  into  the  field 
without  the  elementary  logistics  to  make  them  work. 

This  is  not  to  dismiss  the  progress  made  in  all 
agencies  in  establishing  effective,  efficient:  planning 
systems.    ^!any  of  the  plans  reviewed  during  this  phase 
of  the  evaluation  were  tightly  reasoned,  logical  state- 
ments of  educational  purpose.    Their  teanninology,  though 
it  was  lifted  directly  from  A^^IA,  phrased  administrative 
and  curricular  goals  more  practically  than  the  rhetoric 
used  in  most  other  educational  institutions  we  have 
studied.    Moreover,  even  if  training  was  able  directly 
to  accomplish  little  else,  it  did  raise  peoples* 
consciousness  of  the  existence  of  planning. 


Section  3;    Overall  Conclusions  on  the  Effectiveness 
of  the  MiTi  Program 

Preliminary  Considerations 

Using  the  evidence  and  analysis  presented  in  every 
previous  r-hanter,  we  turn  to  answer  the  critical  question 
of  this  evaluation:    Was  the  AMA  progi.am  worthi^hile?  Did 
the  experimental  educational  agencies  derive  sufficient 
benefit  from  MiA  training  to  justify  the  time  and  money 
it  cost? 


Ibid. ,  p.  4. 
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The  ansvjer  to  this  basic^  general  question  rests 
on  answers  to  subsidiary  questions:     Did  AJIA  training 
positively  affect  the  actions  and  attitudes  of  the 
participating  educational  agencies  toward  management 
and  planning  concepts  and  skills?    A  second^  parallel 
concern  develops:    Even  if  actions  and  attitudes  of 
persons  within  the  organizations  were  improved  by  train- 
ing^ did  these  changes  enhance  organizational  output? 
Did  the  lea's  and  SED's  produce  viable  plans  and  move 
tov/ard  their  stated  educational  objectives? 


Basic  Conclusions 

Two  years ^  a  considerable  investment^  and  the 
cooperation  of  many  sponsoring  and  participating  agen- 
cies^ have  been  devoted  to  these  issues.    Now  the  data 
are  in  and  the  conclusions  have  been  drawn.    V-Tiat  are 
our  overall  conclusions? 

Concerning  actions  and  attitude s^  we  note  that 
all  agencies  have  evidently  committed  themselves  to  some 
degree  of  planning  and  have  produced  planning  documents 
relating  to  the  original  thirteen  AflA  training  goals. 
Despite  this  written  work^  however^  attitudes  about 
planning  and  its  effectiveness  showed  no  change  after 
training.    In  terms  of  the  causal  and  in^3rvening  vari- 
ables^ we  thus  demonstrated  that  training  has  had  no 
significant  effects  on  individual  views  of  organization 
plans  or  internal  environments. 

Given  the  negative  response  to  this  first  question^ 
the  second  question  on  actual  output  becomes  predictably 
academic.    Since  the  AtlA  program  '^id  not  substantially 
change  participant  perceptions  of  plans  or  work  environ- 
ments^ we  cannot  anticipate  any  shift  in  how  much  the 
organizations  were  able  to  accomplish.  Theoretically^ 
output  should  have  remained  relatively  stable;  analysis 
revealed  that  it  did  so.    To  be  sure^  plans  were  developed 
to  establish  department  priorities  and  school  district 
programs.    But  these  plans  failed  to  meet    all  qualita- 
tive criteria  of  viable  plans:    they  required  further 
elaboration  and  some  revision.    In  addition^  the  SED's 
and  lea's  themselves  did  not  believe  that  their  plans 
had  grown  more  opereible  or  that  they  had  progressed  toward 
objectives.    In  view  of  the  inccMnnleteness  of  many  plans # 
as  well  as  the  current  lack  of  reliable  methods  of 
evaluating  organizational  progress^  these  comparatively 
static  attitudes  are  unstartling. 

At  the  conclusion  of  our  study^  the  AMA  program 
had  not  significantly  altered  attitudes  toward  planning^ 
nor  had  it  enabled  the  SED*s  and  LEA's  to  produce  complete^ 
acceptable  plans  or  to  make  measurable  progress  toward 
realizing  them. 
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TheroforGy  the  roscarch  team  concludes  that  the 
7lM7v  program  exerted  no  significant  effect  on  the 
actions »  at^^tudeSy  or  output  of  the  Experimental 
educational    gencies^    Since  virtually  no  significant 
changes  in  the  agencies  appeared,  the  research  team 
believes  that  the  Experimental  agencies  did  not  benefit 
from  AMA  training  to  a  degree  that  justifies  the  time 
and  money  expended  on  the  Progrcun« 


Re  commenda t  ions 


In  our  judgment,  the  A'-IA  planning  process  and 
training  design  demand  considerable  revision.    If  the 
program  were  further  developed  in  a  number  of  aspects 
enumerated  below  (and  introduced  earlier  in  this 
report),  its  impact  i^pon  educational  agencies  might  be 
strengthened.    Whether  the  consequent  improvement  would 
warrant  continuation  of  this  program  or  others  similar  to 
it,  is  a  decision  for  the  reader.    Our  suggestions  are 
directed  toward  two  aspects  of  the  present  AHA  progrcun: 
the  planning  process  and  the  basic  training  design 
itself. 


Issues  Related  to  the  Planning  Process 

Planning  must  be  seen  as  a  flexible  process 
designed  to  meet  the  specific  needs  of  a  given  organiza- 
tion.    Insistence  on  writing  objectives  only  in  terms 
of  student  behavior,  for  example,  may  overlook  critical 
process  goals,  v/hich  in  turn  reduces  the  plan's 
effectiveness. 


The  underlying  assumptions  and  implications  of 
Planning  for  the  particular  organizations  being  trained 
should  be  thoroughly  developed.    Trainees  must  clearly 
comprehend  the  value  and  meaning  of  planning  if  they 
are  to  accept  and  use  it. 12 

The  issue  of  who  writes  the  objectives,  and  for 
what  ends,  must  bo  faced  squarely.    Plans  cannot  effec- 
tively be  imposed  on  people  in  modern,  complex  organiza- 
tions.   As  discussed  in  this  report,  experts  inside  the 
organization  who  are  skilled  in  the  administrative  or 
educational  specialties  covered  by  plans  have  a  critical 
input  to  contribute,  and  they  should  have  every  oppor- 
tunity to  do  so.l^ 


12 

Cf.  Chr:s  Argyris,  Intervention  Thuory  and  Method 
(Reading,  Mass.:    Addison-Wesley,  1971) . 

13 

Q  Warron  Bennis,  Changing  Organizations  (Now  York: 

ERJC  McGraw-Hill,  1965). 
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In  programs  rtesignerl  to  incroaso  planninc,  compe- 
tencG,  the  importance  of  developing  an  unambiguous 
understanding  of  planning  concepts  cannot  be  overesti- 
mated.    The  critical  relationship  of  budgeting  to  ovalua- 
tion  and  nlanninc  should  alsn  be  emphasized.  for..Dlans 
that  rest  on  little  base-line  data  and  carry  no  price 
tags  oniy  veaken  the  credibility  of  planning. 


Issues  Related  to  Training  Design 

The  pre-training  diagnosis  of  organizational 
needs  and  problems  must  be  improved.    Specific  needs 
vary  greatly  among  organizations;  applying  "cookbook" 
solutions  to  all  organizations  as  if  they  were  identical 
is  a  common  error  of  training  designs  such  as  the  one 
evaluated  here.    A  clear  prior  understanding  of  an 
organization's  unique  needs  would  enable  a  training  , 
program  to  be  tailored  to  the  needs  of  the  client  group. 

Management  team  development,  too,  is  a  complex 
process. ^5    Specific  team  development  goals  should  be 
framed,  in  order  to  improve  the  group's  performance. 
This  may  require  approaches  very  different  from  those 
taken  by  strictly  cognitive  training.    Both  the  process 
and  cognitive  elements  of  a  training  sequence  must 
receive  attention  if  genuine,  lasting  organizational 
change  is  desired. 

A  basic  question  of  design  is  the  applicability 
of  Likert's  linking-pin  model  of  organization  to  educa- 
tion.   The  question  of  how  public  educational  organiza- 
tions actually  function  must  be  addressed  within  the 
planning  process. The  imposition  of  a  model  that  may 
not  fit  education  can  prove  extremely  wasteful.  This 
issue  is  relevant  to  the  special  relationships  between 
St?te  Education  Departments  and  Local  Educational 
.Agencies  that  were  discussed  earlier  in  this  report: 
to  assume  in  the  planning  process  that  SED's  control 
lea's,  when  in'  fact  they  do  not,  is  to  render  the  plan- 
ning process  weak  at  the  root. 

A  second  issue  concerns  the  applicability  of 
Likert's  model  to  the  current  individual  role  definitions 


14 

Argyris,  o£.  cit. 

15 

Richard  Beckhard,  Organization  Development 
(Reading,  Mass.:    Addison-Wesley,  1970) . 

^^Ibid. 
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that  exist  in  educational  agoncies.    if  one  is  to 
decide  v^hether  it  is  desirable  to  apply  such  a  model 
to  education,  this  issue  and  its  underlying  assumntions 
analyzed       m  organizations  must  bo  thoughtfully 


r,„4.  4:  suggestions  for  further  development  are  not 

put  forward  as  an  indictment  of  the  agencies,  which 
Mooi''^.''S?^'''^?^y  V®^^  "it^  ^hat  they  had  been  given. 
™  '■^^^^  ^"  program  are  feasible  and 

essential,  even  though  their  implementation  strategy  is 
open  to  serious  criticism. 

This  report  is  intended  to  encourage  the  continu- 
ing evolution  of  the  planning  process  bv  provif'ing 
reliable  information  and  valid  feedback" on  what  has  gone 
before,  in  order  to  improve  what  will  come  later.  For, 
as  has  been  noted  by  another  writer,  "The  purpose  of 
evaluation  is  not  to  prove  but  to  improve 17 ' 


17 

Daniel  L.  Stufflobeam  et  al..  Educational 
l!!^  Decision  Making  (Bloomington,  Indiana! 

Phi  Delta  Kappa,  Inc.,  1971). 
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APPENDIX  A 

SCHEMATIC  PRESENTATION  OF  DATA  -  CAUSAL  VARIABLES, 

CHAPTER  FOUk 
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QSmmt  AWALYSIS  DATA 

Item   1  :  Definition  of  the  Institution's  Mission 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H=  1.218;  Sig .  -  NS 
p=  0.037  ; Sig.*  .05 


Exp. #2 


6. 


H°1.125  ;Sig.  NS 


p ^0.349  ;Sig;  NS 


H-6.942  ; Sig. «.01 


paO.213  ;Sig. 


NS 


Exp.fl 


-o 


H«»  1.422; Sig. »  NS 


Ife  0^507; Sig.  =_NS 


-6 


Exp. #2 


Fall,  1970 

N 

Kruskal-Wallii  One-Way 

Analysis  of  Variance 

Binomial  Test  of 

Experiment  #1 
Experiment  #2 

4 
8 

H= 1.218 
Sig.«  NS 

P*  6.037 
Sig.  .05 

Spring, 1972 

Experiment  #1 
Experiment  #2 

3 
5 

H» 1.422 
Sig.=  NS 

p=  0.507 
Sig.«=  NS 

"  Fall,  l9")0 
Spring, 1972 

Expexiraent  '1 
Experiment  #1 

4 
3 

H«1.25 
Sig.>  NS 

'D=  0.349 
Sig.«  iiS 

Experiment  #2 
<  xperiaent  #2 

8 

5 

H«6.942 
Sig.-  .01 

Sig.=  NS 
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OONTOrr  ANALYSIS  DATA 


Item    2    :  Sense  of  SED  Mission 


SCHEMATIC  PRESBKTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.«l 


p=  0.015  ;Sig. 


Control  #1 


H"  0.000  ; Sig . ■  NS 


H=  0.000  :Sig. ■  NS 

  —Tor 


P»  0.0009: Sig . ■  .001 


H"  0.000; Sig . ■  NS 


0.273;Sig. ■  NS 


Exp.#l 


o 


H'0.000  ; Sig . *  NS 


p  *0.000  :Sig.g  NS 


Control  #1 


Exp. #2 


Fall,  1970  Spring,  1972 

H"  0.000 Sig . -  NS 

0  0 


H=  0.056  :Sig."  NS 
p=  0.000  ; Sig. ■  .001 
002 


Control  ffl 


p ■  0.000  Sig . »  .001 
005 


H-  Q.QQQSig. »  NS 


P  ■  0.273Sig.  ♦  NS 


E3q).#2 


H»  6.000; Sig.  NS 
P  =  0.000;Sig.«  NS 


Control  #1 


ERIC 


Fall,  1970  Spr...g,  1972 

Exp.  #1  H"    O.OOQSig.-  NS 


H»  0.160  ; Sig . ■  NS 


p*  0.028  : Sig . -  .05 


Exp. #2 


p »0.0009  ; Sig . ■  .001 


H"  0.000  ;Sig. «  NS 


p«  0.  000  ;Sig.«   . 001 

— mr   


Exp.#l 


N£ 


Hg  0.00O:Sig.« 
p  =  0.000;Sig.a  NS 


6 


Exp.«2 


Fall,  1970 

N 

Kiriiskal-lfAlli«  nitM-Vow 
Analysis  of  Variance 

Binominal  Test  of 
Proportions 

Control  #1 

H»  0.000 

P  «  0.015 
Signif.-.  01 

2 

Signif.B  NS 

Experinent#2 
Control  #1 

Q 

7 

u  .  U  3  o 

p  -  0. 000002 
Signif.-.  001 

2 

Signif."  NS 

Experiment#l 
Experi]Dent#2 

5 

H»  0.160 

p«0.028 

9 

Signif.-  NS 

Signif.-.  05 

Spring,  1972 

•- 

Experimenttl 
Control  #1 

1 

H»  0*000 

p-0.000 
Signif. -NS 

2 

Signif.>  NS 

Experiment#2 
Control  #1 

2 

Hb  0  onn 

si"    \J  m  \J  \J  \J 

n     A  AAA 
P=U  .  000 

Signif. -NS 

2 

Signif.*  NS 

Experiment #1 
Experiment#2 

1 

H-  0.000 

P*0.000 
Signif.  =  NS 

2 

i 

Signif.-  US 

Fall,  1970 
to 

Spring,  1972 

• 

• 

Experiment  #1 
Experiment  #1 

5 

H»  0. 000 

p=0.0009 
Signif. -.001 

1 

Signif. -NS 

cAp^rxmonc 
Experinent  #2 

q 

}{m  U  •  U  U  U 

p=  0. 000005 
Signit»-001 

2 

Signif.-  NS 

Control#l 
Control#l 

.  2 

O.ODO. 

P  =  0.273 
Signif.-  NS 

2 

Signif.*  NS 
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CONTENT  ANALYSIS  DATA 


Item    3    :  How  do  you  feel  about  the  direction  your  organization 


IS  moving.-' 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 


Fall.  1970 


Exp.fl 


H=  0.187  ■  Sig."  NS 
p=  0.000  : Sig . *  NS 


Control  #1 


H»  0. 126  ;Sig."  NS 
P«  0.000  ; Sig . «  NS 


H»  1.5"l:  Sig . »  NS 


p»  o.non;Sig.»  ns 


Spring,  1972 

 0 


Exp.#l 


H=iL^25_:Sig.»__Ns_ 
P *0.000  : Sig .  =  NS 


Control  #1 


ERIC 


Fall,  1970 


Exp.*2 

Hg6.307  : Sig . ■  .05 
P=Q.OQ0  :Sig.»  


H»  0. 004 Sig . »  NS 


Control  #1 


p '  0.000 Sig . «=  NS 


Spring,  1972 
 Q  Exp. #2 


H-  1.591  Sig.  =  NS 


H"2.083  ; Sig. NS 
p  =0.000  ;Sig.=-.  NS 


Control  #1 


p«  Q.OOOSig.*  JiS. 


Fall.  1970 


Exp.  #J. 


H«  0.126  ;Sig. 


NS 


Spring,  1972 

 0 


Exp.#l 


H»7.680  : Sig. »  .01 


p»0.  000  :Sig. «  NS 


p»   0.000  ;Sig. a 


NS 


H»0.0C1  ;  Sig .  =  NS 


H=4.335  : Sig . "  .05 
p  =  0.000  ;Si8.=  NS 


Exp. #2 


p sO.COO  ; Sig .  =  NS 


0 


Exp. #2 


ERIC 


474 


Itm  3 


'  How  do  you  feel  about  the  direction  ynnr  r.^rj.r.i  ..^^^j. 
IS  moving?   '  2  


Experiment#2 


12 


12 


Spring,  1972 


Experiment#l 
Control  #1 


Experiment#2 
Control  #1 


11 


10 


11 


Experimenttl 
Experiment#2 


10 


Fall,  1970 
to 

Spring,  1972 


Experiment  #1 
Experimenc  #1 


12 


U. 


Experiment  #2 
Experiment  #2    |  lo 


Control#l 
Control#l 


12 


11 


H»  7.680 
Signif.>  .01 


H»  0.975 
Signif.-  ^'s 


H»  2.083 
Signif.*  NS 


H-  4.335 


Signif."  .05 


H- 0.126 


Signif.  >  NS 


H"  0.004 
Signif. »  NS 


H-  1.591 


Signif.-  NS 


Fall,  1970 

N 

Analysis  of  Variance 

Binominal  Test  of 
Proportions 

Experiment #1 
Control  #1 

12 

H*  0  187 

1          P  -  0 . 000 
Signif.-  NS 

12 

Signif.-  NS 

Experiment#2 
Control  #1 

12 

H-6.307 

p  ■  0. 000 

12 

Signif.-  .05 

Signif.-  NS 

P»  0. 000 
Signif.  -  NS 


P=0.000 


Signif.=NS 


P=0.000 


Signif.  =  NS 


P*0. 000 


Signif.  =  NS 


p=0. OUO 
Signif.-  NS 


P"  0.000 
Signify?  NS 


P  »  0.000 


Signif..  NS 
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QUESTIONNAIRE  DATA 


Iteqj  4_    The  kinds  of  things  I  am  doing  will  make  a  long-term  contribution 

to  education.  .  ' 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 

Total  


Two-Wav  Anal  of  Var. 


Experiment  SED#1  W/ 
Control  SED  #1  


Fall,  1970 

T1 


N 

39 
60 

I! 

165 


X 

■■7.43fi 
5.467 

5.697 


SD 


1.214 


1.301 


Spring,  1971 


N 

73 


51 
61 


Col, 


Tl    6  T2 


0.005 


0.096 


 tS 


X 

5.630 

SD 
1.339 

N 

68 

■  X 

5.838 

5.196 

1.233 

45 

5.200 

5.393 

1. 159 

67 

5.507 

ST 

5.^25 

T2    6  T3 


0.959 


3.232 


■Signif  ■ 


NS 


NS 


Fall,  1971 


SD 
1.31 


"ills 
1.106 

rro? 


JZ  6 


2^134 


4.160 


T4 


NS 


.05 


Spring,  1972 
-14- 


N 

JO 
40 
61 


207 


2761 


SD 
.051 


X 

5.60011 
£.97511.423 
5. 


.25; 


Tl 


6  T4 


0-560 


0.060 


NS 


NS 


Experiment  SED#2  W/ 
Control  SED  #1 
Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


CoL 


1.785 


0.074 


NS 


1.587 


NS 


6.694 


Row 


3.  217 


NS 


2.642 


NS 


2.  879 


NS 


2.  288 


NS 


Col, 


Row 


1.817 


NS 


NS 


CoL 


Row 


1.340 
6.324 


NS 


Tor 
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QONTBNT  ANALYSIS  DAT*, 

I tem  5  ;  Modify  previously  established  objectives. 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H=  0.028:  Sig . «  NS 
p  =  0.03  : Sig . «  .05 


74 
Exp. #2 


6_ 


H"  1.508  : Sig .  NS 


D  9  0.000  ;Sig;  .01 

^  —  


H»  3.891  ; Sig . -  .  05 


P "  ^-017  ; Sig."  .  05 


Exp.«l 


H-1.750  ;Sig.«  Ni 
1^0.450  ;Sig.=  Nl 


d 


Exp. #2 


Fall,  1970 

N 

Kruskal-Wallii  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

PrfiprtTtinn«s 

Experiment  #1 
Experiment  #2 

4 

H=  0.028 

P»  0.0374 
Sig.  .  05 

8 

Sig.«  NS 

Sprinq,  1972 

Experiment  #1 
Experiment  #2 

7 

H»  1.750 

p=0.450 
Sig.=  NS 

9 

Sig.=  NS 

Fall,  1970  - 
Spring,  1972 

Experiment  #1 
Experiment  #1 

4 

H=  1.508 
Sig.«  NS 

-D=  0.000966 

7 

Sig. »  .01 

Experiment  #2 
xperiment  #2 

8 

H"  3.  891 

p«  0.017 
Sig. =  .05 

9 

Sig.».05 
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COWrafT  AHALYSIS  DATA 

Item  2   ;  Identify  and  analyze  alternative  courses  of  action. 


SCHEMATIC  PRESEHTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H=  1.256;  Sig . «  NS 


P=  0.774:  Sig. «  NS 
Exp. #2 


o. 


H"  1.442  ; Sig.  NS 


P  "  0.  089  ;Sig;  NS 


H«  2.564]:Sig.«  NS 


.       0.171  ;Sig.»  NS 


o 


Exp.#l 


H«3.102  ; Sig . «  NS 
I^iL^aiI.:Sig.=_ii£. 


-6 


Exp.«2 


Fall,  1970 

N 

Kruskal-Wallii  One-Way 

Analysis  of  Variance 

Binomial  Test  of 

 BasftpArt.i  ntic 

Experiment  #1 
Experiment  #2 

6 

H«  1. 256 

P*  0.774 

6 

Sig.>  NS 

Sig.  NS 

Sorina.  1972 

Experiment  #1 
Experiment  #2 

6 

H-  3.102 

P»  0.880 

6 

Sig.»NS 

Sig.«  NS 

Pall,  1970- 
Spring,  1972 

Experiment  #1 
Experiment  #1 

H-  1.442 

n.0B9 

6 

Sig.» 

Sig.-  tis 

Experiment  #2 
;q>eriment  #2 

6 

H-  2.5641 

P'  0.171 

6 

Sig."  NS 

Sig.«  NS 

ERJC 
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QOKTENT  ANALYSIS  DATA 


^ ten  3    ;  Determine  priorities. 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Spring.  1970  Fall,  1972 


Exp.n 


H=0.949  tSig. «  NS 
p =0.378  ; Sig. *  NS 


Exp. #2 


H«4.408  ; Sig .  .05 


p    0.066  ;Si£;  NS 


H«  5.485  ; Sig. ■  .02 


p  =  0.000  :Sig.»  .001 

— mr   


Exp.#l 


P^O.OOO  sSig .  =  NS 
002 


Exp. #2 


Spring,  1970 


Kruskal-Wallii  One-Way 

Analysis  of  Variance 


Binomial  Test  of 


Experiment  #1 
Experiment  #2 


H=  0.949 


Sig.»  NS 


P*  0.378 


Sig.  NS 


Fall,.  1972 


Experiment  #1 
Experiment  #2 


10 


Sig.*  NS 


Sig. a  .  001 


Spring,  197' 
Fall,  1972 


Experiment  #1 
Experiment  #1 


H=  4.408 


Sig.».05 


0.066 


Sig.«  NS 


Experiment  #2 
xperiment  #2 


10 


H»  5.485 


Sig.  ■•02 


p=  0.000001 
Sig. «  .001 
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Item  4 


CXafTEHT  ANALYSIS  DATA 


Do  you  feel  that  the  objectives  developed  as  a  result  of  AMA 

training  reflect  the  most  sftrious  and  prossina  n^^Ms  of  state 
education?  «ta<L_sc_it*_buu*j.i«- 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1971  Spring.  1972 


Exp.ffl 


H=  5.041;  Sig .  -  .05 
p  =  1.000; Sig .  *  NS 


Exp. #2 


H-0.124;Sig.  NS 


p  ■»  1.000  :Sig;  ns 


H- 0.142  ; Sig. ■  NS 


p al.OOO  : Sig. «  NS 


Exp.n 


"O 


H°2.5^s  :Sig.« 


P^l.OOO  : Sig .  =  NS 


-6 


Exp. #2 


Fall,  1971 

N 

Kruskal-Nallii  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Experiment  #1 
Experiment  #2 

9 
10 

H«  5.041 
Sig.-  .^5 

P •  1.000 

Sig.  NS 

Spring,  197 

2 

Experiment  #1 
Experiment  #2 

9 
10 

H. 2.535 
Slg.»  NS 

P=  i.non 
Sig.=  NS 

Fall,  1971- 
Sprii>g,  197 

I 

Experiment  #1 
Experiment  #1 

9 
9 

H»  0.124 
Sig."  NS 

•P-  1.000 
Sig.»  NS 

Experiment  #2 
xperiment  #2 

10 
10 

H- 0.142 
Sig.-  NS 

p= 1.000 
Sig.=  NS 
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QUESTIONNAIRE  DATA 


Iteiii  5__      The  goals  of  this  organization  are  articulated. 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2' 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Two- Way  Anal  of  Var. 


Fall,  1970 

T1 

N 

X 

SD 

163 

4.203 

1.648 

5.231 

1.222 

Spring,  1971 
_X2. 


N 


52 
61 


186 


Tl    6  T2 


X 

4.fin8 


4.096 


4.496 


SD 

1.376 
1.501 


Fall,  1971 


N 

68 


_45 

67 


T2    6  T3 


X 

5.41]]1 


SD 

.318 


4.8221.402 
4.8951.232 
4.737ll.662 


T3 


\  T4 


Spring,  1972 


N 

40 


39 


59. 


204 


X 

5.175 
4.256 


4.610 


4.545 


SD 
1.2713 
1.463 

1.453 
1.337 


6  T4 


Experiment  SED#1  W/ 
Control  SED  #1 


CqL 


0.197 


7.578 


NS 


8.80 


i 


.01 


.369  . 
.42J  . 


.01 


2.175 


■Signif 


VS 


1.782 


05 


9.328 


.1)1 


1.400 


NS 


Experiment  SED#2  W/ 
Control  SED  #1 


14.^19 


Row 


5.063 


.05 


1.70a 


01 


NS 


4t7??. 


1.207 


.05 


1.856 


NS 


NS  HL.665 


.001 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


Col, 


Row 


1.290 
11. §14 


NS 


Col. 


3.327 


NS 


Row 


0.242 


NS 
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QUESTIONNAIRE  DATA 


Itei|L_6  Our  goals  are  realistic  and  attainably,  ^i^-h  ^^y:    besfc  ^ffnrfs. 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3. -6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


Fall,  1970 
.JLL 


N 

38 

•55 
"B 

16  3| 


X 


4.610 
5.379 


SD 


1.092 


Spring,  1971 


N 

61 
1185 


X 

4.94fi 


4.392 


4.902 


So 


1.297 
1.589 


Fall,  1971 


N 
45 


67 

6T- 


241 


X 


5.133 
5.507 


5_aS3 


SD 


N 

-40 

 61 


i.2r 

1.06i 


spring,  1972 
-S4. 


SD 

5.2Qq-LJl 


5.2i: 


5.33 


4.75(  1.3:i4 


l.lJi6 


j^(;7 


Two-Wav  Anal  of  Var. 

Tl    S  T2 

T2    &  T3 

T3  6 

T4 

Tl    6  T4 

Experiment  SED#1  W/ 
Control  SED  #1 

Signify 

P 

Signif . 

P 

V 

Signif.  ,  ., 

0.000 

NS 

8.262 

to; 

1.221 

NS 

n.ftns 

NS 

6.288 

.05 

0.186 

NS 

o.fi2n- 

NS 

O.OOfi 

NR 

Experiment  SED#2  W/ 
Control  SED  #1 

11.964 

.001 

15.491 

.001 

4.218 

.05 

n.nn? 

NS 

Row 

3.539 

NS 

5.681 

.05 

6.439 

.05 

11.457 

.001 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

0.285 

NS 

Row 

0.845 

NS 

Col. 

1.919 

NS 

Row 

6.120 

.05 
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QUESTIONNAIRE  DATA 


Item  7 


The  top  priority  objectives  of  state  education  are 
clear  to  me. 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


SI 


Fall,  1971 
T3 


•a 


Spring^^l972 


7.0 
6.8 
6.6 


5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
8 
6 
4 
1.2 
1.0 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


N 

45- 
62. 

51. 


X 

^(il7 
5.4 

t: 


]Qpi. 

nr. 

WsT. 


.  SD 
11.425 


575 
T95 


N 
40 

40 

61 


ML 


5.40d 


4.725 


4.96t 


1.29 
17^ 


1.38 


^Iwo-Wav  Analysis  n 

f  Yar. 

Experiment  SED#1 
WAZontrol  SPn#l 

2.652 

NS 

Experiment  SED#2 
W/Control  SPn#i 

rni 

1.437 

NS 

5.232 

.05 

Experiment  SED#1 
W/Contro1  SPn#7 

Cnl. 

2.151 

NS 

.&QW 

4.136 

Experiment  SED#2 
W/Control  SPn#9 

Col. 

6.§4§ 

MS 

Row 

3.241" 

NS"  ■ 
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QUESTIONNAIRE  DATA 


Item   8      I  feel  that  the  objectives  developed  during  AMA  training 

reflect  the  innsf  sprimis  anH  nr^gaina  n^^Hs  of  <ii-^^ -riun^i-inn , 


Scale 
Values 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
S.8 
5.6 
5.4 
5.2 


3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
•2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Two  Way  Analysis  of 
Variance 


Experimental  SED#1 
W/Experimental  SED#2 


Fall,  1971 
T3 


Fall,  1971 
T3 

Spring,  1972 
T4 

N 

X 

SD 

N 

X 

sb 

Experimental  SED#1 
Experimental  SE0#2 
Total 

4.926 

i.aR? 

5.333 

1.131 

45 

4.333 

1.381 

39 

4.178 

1.555 

113 

78 

Spring_j_^1972 


7.0 
6.8 
S.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
'.'.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


T,  5  T, 


;ai. 


low 


0.024 


Signif. 


NS 


.001 
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QUESTIONNAIRE  DATA 


Item    9      As  I  see  it,  the  operational  priorities  of  the  objec- 
tives developed  during  AMA  training  are  clear. 


Scale 
Values 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
•2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


JL.. 


Fall,  1971 
T3 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  SED#1 
Experimental  SED#2 
Total 

N 

X 

4.  5'iR 

SD 
1.887 

N 

37 

X 

5.108 

SD 
1.264 

45 

4.200 

1.501 

39 

3.589 

1.516 

113 

76 

Two  Way  Analysis  of 
Variance 

Experimental  SE0#1 
W/Experimental  SED#2 

F 

Sienif . 

0.015 

NS 

.001 

Spring.  1972 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 
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OOWTBfr  AWAIYSIS  DATA 
Iten  1  :    Define  standards  of  performance  for  key  administrators. 

SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H=  0.600:  Sig .  -  NS 


P  =  0.644:  Sig .  •  NS 


Exp. #2 


H«  1/777  : Sig.  NS 


p 'yO.OOO  ;Sig:  .001 
095 


H» 0.680  ; Sig. »  NS 


p "0.000  ; Sig. ».ooi 

001 


Exp.n 


"O 


H"0.003  rSig."  n 
^=0.104  ;Sig.=  N 


o 


Exp. #2 


Fall.  1970 


Kruskal-Wallii  One-Way 

Analysis  of  Variance 


Binomial  Test  of 
 Rmpnrtinns — 


Experiment  #1 
Experiment  #2 


H=  0.600 


Sig.«  NS 


P*  0.644 
Sig.  NS 


Spring,  197  2 


Experiment  #1 
Experiment  #2 


.  Sig.«  NS 


P°  0-104 

Sig."  NS 


Pall,  1970- 
Sprinq.  1972 


Experiment  #1 
Experiment  #1 


H« 1.777 
Sig."  NS 


>p'  0.000095 


Experiment  #2 
xperiment  #2 


H»  0.680 
Sig."  NS 


p"  0.000001 
Sig. 001 
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QOWTBWT  AWALYSIS  DATA 

I  tea  2   :    Specify  task  completion  dates  and  action  assignments. 


Exp.  #1 


SCHEHATIC  PRESBtTATIW  OF  ANALYSIS 

Fall,  1970  spring,  1972 

He  1.562  ;Sig.  NS 


Exp.#l 


H=  1.800:  Sig.  ■  NS 


p=  0-316.sig..  NS 
Exp. #2 


p"  0.000  :Sit;  .001 
001 


6. 


H»  3.125  t Sig. «  NS 


H»3.681  ;  Sig.  =  NS_ 
P^O.152  :Sig.=NS 


p a  0.078  ;Sig.»  NS 


-6 


Exp. #2 


Fall,  1970 

N 

Kruskal-Wallii  One-Way 

_   Analysis  of  Variance 

Binomial  Test  of 

—PmpfiTtinnc 

Experiment  #1 
Experiment  #2 

1 

H=1.800 

P •0.316 
Sig.  NS 

3 

Sig.»  NS 

Spring,  1972 

Experiment  #1 
Experiment  #2 

6 

H»  3.681 

P  =0.152 
Sig.a  NS 

4 

Sig.»  NS 

Fall,  1970- 
Spring,  1972 

Experiment  #1 
Experiment  #1 

1.562 

-P '0.000001 

6 

Sig."  NS 

Sig.  =  ,001  . 

Experiment  #2 
xperiment  #2 

\ 

H-  3.125 

p '0.078 

4 

Sig.«  NS 

Sig.=  NS 
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QcnfiBrr  analysis  data 


I tern  3  ;  Assign  resporsibilities  to  subordinate  units. 


SCHEMATIC  PRESENTATKm  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H=0.083  : Sig .  =  ns 


p=0.780  : Sig. «  NS 
Exp. #2 


6_ 


H»  2.142; Sig.  NS 


p  t  0.000  ;Sig;  . 001 
005 


H«  4.521  ;Sig.n>05 


D "  0.000  :Sig.-»^  .GGi 
001 


"O 


Exp.«l 


"••2.Q42  -S^g-'ilg- 
P-Q.45Q  ;Sig.  =  NS 


Exp. #2 


Fall,  1970 


Experiment  #1 
Experiment  #2 


KToskal-Wallii  One-Way 

Analysis  of  Variance 


H=  0.083 


Slg.= 


NS 


Binomial  Test  cf 


P  *  0.780 


Sig.  i:s 


Spring,  1972 


Experiment  #1 
Experiment  #2 


H»  2.042 
Sig.  a  NS 


?=_iUA5SL 
Sig.«JiS_ 


Fall,  1970- 


Experiment  #1 
Experiment  #1 


Hb  2.142 


Sig.-  NS 


•P'  0.000005 
Sig. «. 001 


Experiment  #2 
xperiment  #2 


H»  4.521 


Sig.«  .05 


p«  0.000001 
Sig. «. 001 


1  tp;a  Z 
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QUESTIONNAIRE  DATA 
My  organization's  policy  statements  are  c.lpar. 


SCALE 
VALUES 


7.0 
6.>i 
6.0 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 


3, 
3. 
3, 
3, 
3, 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


[Experimental  SOD  #1 
Experimental  SED  #2 
Control  SED 
Control  SED  #2 
-  Total  


Two-Way  Anal  of  Var. 


Fall,  1970 
T1 


N 

3S 
59 
_66 

163 


X 

5.132 


4.237 
5.500 


SD 
1.  547 
1.622 
1.113 


Spring,  1971 
T2 


N 
71 


51 


57 


Tl    f,  T2 


X 

5.  239 


SD 

  1.439 

4.020  1.378 
5.298  1.322 


T2    6  T3 


Fall,  1971 
--X3. 


N 
68 


45 


67 
6T 


2"4T 


X 

5.176 

4.^44 
4.912 
4..6Q6 


SD 

N 

\ 

.257 

4J1 

1 

.264 

39 

1 

.  100 

59 

1 

.666 

66 

T3  6 


T4 


Spring,  107: 


204 


X 

4.230 
4.813 
4.439 


6.5 


J  : 


!1. 

M. 


i4.. 

i 

I*- 

p.o 

.1 


2.C 

1.0 

1.-' 

1.-. 


51) 

1^18 
1.441 

_l_^j7b 
1.  34H 


Tl    6  T4 


Experiment  SED#1  W/ 
Control  SED  #1 


Col, 


Row 


0.066 


Signif 


NS 


1.183 


NS 


0.267 


Signi  f ■ 


NS 


0.244 


Signi  f ■ 


NS 


2.919 


NS 


2.915 


NS 


0.019 


■Signif, 


NS 


NS 


Experiment  SED'*2  W/ 
Control  SED  ^'1 


1-071 


-US- 


3-471 


21.457 


001 


fi.726 


01     23.340  j  ,0J11 


Experiment  SED*1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


Col, 


Row 


0.150 
12.894 


NS  ■ 

jJoT 


Col. 


2.026 


NS 


Row 


0.175 


NS 
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QUESTIONNAIRE  DATA 


My  organization's  performance  standards  are  understood. 


SCALE 
VALUES 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


« 
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QUESTIONNAIRE  DATA 


_9       Good  ways  are  used  to  let  me  know  how  I  can  improve  my  performance. 


SCALE 
VALUES 


5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.? 

4.0 

3.8 

3.6 

3.4. 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Fall,  1970 

Tl 


Spring,  1971 

T7 


Fall,  1971 


•Spring,  1972 

TA 


7.0 
6.8 

.  6 

6.2 
6.0 


8 
6 
4 
-> 

,0 
.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1  .0 


Lxperimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


N 

19 
60 
J5 

164 


X 

4.743 
3.666 
4.707 


SD 
1.229 


N 

73 


1.612 
1.588 


52 


64 
189 


X 

4.575 


3.  346 


4.015 


SD 
1.342 


N 

68 


X 

4.558 


1.311 


45 


1.768 


67 


61 


3.511 
4.149 
3.934 


SD 
1.529 

1.471  39 


N 

40 


1.57E 


60 


66 


241 


205 


Two-Wav  Anal  of  Var. 

Tl    5  T2 

T2    8  T3 

T3  G 

T4 

Tl    6  T4 

Experiment  SED#1  W/ 
Control  SED  #1 

F 

Si£nif . 

F  , 

P 

F 

C9l. 

4.569 

.05 

0.096 

0.36" 

1.473 

NS 

2.190 

NS 

6.589 

"  .05 

3.629 

NS 

0.872 

NS 

Experiment  SED#2  W/ 
Control  SED  #1 

QqI. 

6.057 

.05 

0.509 

NS 

0.378 

NS 

1.240 

NS 

Row 

17.286 

.001 

9.769 

.01 

10.184 

.01 

L7.529 

.001 

Experiment  SED#1  W/ 
Control  SEU  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

0.097 

NS 

Row 

10.343 

.01 

Col. 

0.108 

NS 

Row 

3.381 

NS 

491 

QUESTIONNAIRE  DATA 


Item  10        I  understand  what  results  must  be  produced  to  achiev^e 
the  stated  objectives  of  this  organization. 


Scale 
Values 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


Two-Wav  Analvsis  o 

f  Var.. 

i  T4 

Experiment  SED#1 

W/roTitrnI  SF.n#1 

p 

rni 

NS 

NS 

Experiment  SED#2 
W/Control  SPn#1 

rm . 

l.fi73 

NS 

Rnu 

i.413  . 

NS 

Experiment  SED#1 

W/rontrol  SRn#7 

7.fiq:^ 

.01 

IRow 

1.802 

Experiment  SED#2 
W/Control  SPn#2 

Col. 

^.907  , 

NS 

Row 

9.972 

US 

7  0 

6  8 

6  6 

6  2 

6  0 

D  •  O 

^  •  VI 

tti 

*>  •  ^ 

5  0 

*t  •  o 

4  6 

*T  •  V* 

CI 

4  4 

4  2 

4  n 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4  . 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall,  1971 
T3 


N 

67 
|27I 


X 

5.397 
4.688 
4.955 


1.4571 


1.427 


1.35: 


Spring^^l972 


N 

40 
40 
60 

^75" 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


ERIC 
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QONTBOT  AHALYSIS  DATA 


Item  1  :    Role  of  Planning  Unit  -  Awareness  of  need  to  evaluate  our 
programs . 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1971  Spring,  1972 


Exp. #1 


H= 


:Sig.-  - 
:Sig.»  - 


Exp. #2 


Cr 


6_ 


H"  1.500:Sig.  NS 


p     0.068;Sig;  NS 


-  :Sig.< 


P". 


:Slg.< 


Exp.#l 


H"  0.750  tSig.'NS 
0.924  tSig.^NS 


6 


Exp. #2 


Fall,  1971 

N 

Kruskal-Wallil  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Prnj»nrt.inn<;_  . 

Experiment  #1 
Hxperiment  #2 

0 

P  •  - 

0  . 

Sig.=  - 

Sig.- 

Sprinq,  197: 

\ 

Experiment  #1 
Experiment  #2 

2 

H»  0.750 

p  =  0.924_ 
Sig.=  NS 

3 

Sig.»  NS 

Fall,  1971- 
Spring,  197: 

Experiment  #1 
Experiment  #1 

1 

H«  1.500 

'p-o.oes 

Sig.»  NS 

2 

Sig.-  NS 

Experiment  #2 
xperiraent  #2 

0 

H- 

3 

Sig.«  - 

Si£.=  - 
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QOWTEWT  ANALYSIS  DATA 

I tem  2   ;  Role  of  Planning  Unit  -  Available  to  answer  planning  questions. 

SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1971  Spring,  1972 


Exp.#l 


H=  0^600  :Sig._NS 


Exp.fl 


H=  4.59i:Sig.»  .05 


p=  0.924;Sig.«  NS 


Exp. #2 


p  "  0.026  :Sig;  .05 


'O 


o. 


H"  3.050  t Sig . «  NS 


H'l.lBl  :Sig.".01 


p  a  0.  028  ; Sig. "  .05 


o 


Exp. #2 


Fall.  1971 

N 

Kruskal-Walli&  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Pmpnrtinn<5 

Experiment  #1 
Experiment  #2 

6 

H=  4.591 

P«  0.924 

7 

Sig.=-  .05 

Sig.  NS 

Sorina.  1972 

Experiment  #1 
Experiment  #2 

R  . 

H»  7.787 

P=  n-774 

9 

Sig.»  .01 

Sig.»  NS 

Pall,  1971- 
Spring,  1972 

Experiment  #1 
Experiment  #1 

6 

H»  0.600 

.p«  3.050 

8 

Sig.«  NS 

Sig.-  NS 

Experiment  #2 
•  xperiment  #2 

7 

H«  0.026 

p»  0.028 

9 

Sig.«  .05 

Sig. «  .05 
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QOWTBWT  AWALYSIS  DATA 


Item  3 


Role  of  Planning  Unit  -  Writing  guidelines  for  plan 
development.  "  " 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1971  Spring,  1972 


Exp.#l 


H=  3.266;  Sig .  =  NS 


P  =  0.038: Sig . *  NS 


Exp. #2 


6. 


H''0.006  ;Sig.  NS 


p   0.377  ;Sig;  NS 


H»4.500  ; Sig. «  .05 


p  30.150  ; Sig. ■  NS 


Exp.#l 


H"6.  54  5  ; Sig. a. 02 


-6 


Exp. #2 


Fall,  1971 

N 

Kruskal-Wallii  One-Way 
^lYsis  of  Variance 

Binomial  Test  of 

 Pynpn■r«^^nng 

Experiment  #1 
Experiment  #2 

6 
3 

H=  3.266 
Sig.=  NS 

p •  0.038 
Sig.  NS 

Spring,  197 

2 

Experiment  #1 
Experiment  #2 

6 
4 

Sig.  =  .02 

Sig. a  NS 

Fall,  1971- 
Spring,  197 

2 

Experiment  #1 
Experiment  #1 

'6 
6 

H=  0.006 
Sig.«  NS 

-p»  0.377 

Sig.r  NS 

Experiment  #2 
xperiment  #2 

3 
4 

H«  4. 500 
Sig.«.05 

p=  0.150 
Sig.«  NS 
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QWffBWT  iWALYSIS  DATA 

I  tea  4  ;  Role  of  Planning  Unit  -  Reviewing  and  refining  plans. 

SCHBMATIC  PRESENTATION  OF  ANALYSIS 
FflU.  1971  Spring.  1972 


Exp.#l 


H=  0.085;  Sig.  ■  NS 


p =0.0002;  Sig .  •  .001 
Exp.*2 


6. 


H«  0.771  ;Sig.  NS. 


D«  0.000  :Sig:  .001 


H<*  0.675  ; Sig. ■  NS 


p «  0. 171  ; Sig. ■  NS 


Exp.'l 


H"4.033  ;Si8.». 
P^O.964  :Si8.»  i 


■O 


Exp. #2 


Fall,  1971 


Kruskal-Wallif  One-Way 
Analvgjg  of  Variance 


Binomial  Test  of 


Experiment  #1 
Experiment  #2 


H"  0.085 
Sig."  NS 


P*  0.002 
Sig-  tOOI 


Spring,  197 


Experiment  #1 
Experiment  #2 


6 


Sig.«.05 


Sig.g  NS 


Pall,  1571 
Spring,  197i2 


Experiment  #1 
Experiment  #1 


5 


H«  0.771 


Sig."  NS 


.p»  0.000118 
Sig."  .001 


Experiment  #2 
xperiment  #2 


H"  0.675 
Sig."  NS 


p"  0.171 
Sig."  NS 
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OONTBNT  ANALYSIS  DATA 


I  tern  5 ;    Role  of  Planning  Unit  -  Provides  leadership  in  the 

 implementation  of  planning.  


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall.  1971  Spring,  1972 


Exp.#l 


H=  1.408; Sig. «  NS 


p  =  0.964; Sig . «  NS 


Exp. #2 


H"  1.114  :Sig.  NS 


p  ^0.141  tSig;  NS 


"■.2>4Q8,-Slg."jiS. 


p- 0.366  ;Slg.«NS 


Exp.#l 


H"3.487  iSig.^NS 
P"0.098  ;Sig.=NS 


6 


Exp. #2 


Pall,  1971 

N 

Kruskal'Wallii  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Experiment  #1 
Experiment  #2 

5 

"=  1.408 
Sig.°  NS 

P*  0.964 
Sig.  NS 

Spring,  1972 

Experiment  #1 
Experiment  #2 

7 

H»  3.487 
Sig.»  NS 

P=  0.098 
Sig.=  NS 

Fall,  1971- 
Sprinq,  1972 

Experiment  #1 
Experiment  #1 

5 
7 

H"  1.114 
Sig.-  NS 

-D"  0.141 
Sig.«  NS 

Experiment  #2 
xperiment  #2 

6 

5 

H»  2.408 
Sig.-  NS 

p»  0.366 
Sig.=  NS  , 
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QOWTBWT  AHALYSIS  DATA 


I  tem__6_:  Role  of  Planning  Unit-.  ~  PrnviHps   i  n-g^rv,        ^-r-ainino  in 

Planning.  


SCHEMATIC  PRESBfTATION  OF  ANALYSIS 

Fall.  1971  Spring.  1972 


Exp.#l 


H«  0.750  tSig.  NS 


Exp.tl 


Hg  1.125;  Slg .  ■  NS 


P  =  0.964.- Sig. *  NS 


Exp. #2 


6. 


H«  : Sig. «  MR 


H''2.40Q  ;Sig.  =  NS 
P*0.678  ;Si8.  =  NS 


p  -  0.167  ; Slg. «  NS 


Exp. #2 


Fall,  1971 

N 

Kruskal-Ktllli  One-Nay 
Analysis  of  Variance 

Binomial  Test  of 

Pa!rtpttTtinn« 

Experiment  #1 
Experiment  #2 

,  .a., 

4 

H-  1.125 
Sig.«  NS 

P»  0.964 
Sig.  , 

Spring,  197 

2 

Experiment  #1 
Experiment  #2 

2 

2 

H«  2.400 
Sig.«  NS 

P»  0.67fl 
Sig.«Ns 

Fall,  1971- 
Sprinq.  197 

2 

Experiment  #1 
Experiment  #1 

3 
2 

H«  0.750 
Sig.«  US 

'D»  n.i77 
Sig.- 

Experiment  #2 
xperiment  #2 

4 
2 

"■3.428 

Slg.>  NS 

?■  n.m. 

Sig."  >jp 
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QUESTIONNAIRE  DATA 

Icerr.     7         The  Plfjn"^""  nnit-   has  h«>An   h«.1pfn1  mp . 


Scale 
Values 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
■2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


Fall,  1971 
T3 


N 

67^ 

24]J 


X 

4.867 
3.644 


3.746  1 
3.737ir 


1.93q 


1.73/ 


.77q 


Spring^^l972 


N 

38 
40 

53 

■a 


X    TsD  ^ 


3.750 


3.849 


1.497 


1.472 


1.46 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Two-Wav  Analysis  o 

F  Var. 

Experiment  SED#1 
W/flnnfrnl  SFn#l 

P 

fnl . 

HS  - 

.nm  1 

Experiment  SED#2 
W/rontrol  SF.n#l 

Cnl.., 

Q.12& 

NS 

Rnw 

0.184 

NS  . 

Experiment  SED#1 
W/rontrn1  SPn#2 

rnl,.. 

0.712 

N&  .. 

23.185 

.001  ^ 

Experiment  SED#2 
W/rontrol  SRn#2 

Col. 

0.007 

NS 

Rott... 

0.019 

NS 
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QONTBlff  AHALYSIS  DATA 

I  tea  1  ;  Designed  a  methodology  by  which  future  performance  may  be  

evaluated  in  relation  to  the  performance  specitiea  m  tne  plan. 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall.  1970  Spring,  1972 


Exp.*l 


H= 3.5714Sig."NS 
P  =  0. 144  : Sig . *NS 


Exp. #2 


6. 


6.545!Sig.  .02 


p  »  Q.OOSiSig;  .01 


H"  8.076; Sig . ■  .01 


p"  Q.4Q9:Sig."  NS 


Exp.*l 


H«  1.200  ;Sig.»  NS 
0  42fi  :Sig . '  NS 


Exp.  92 


Fall.  1970 

N 

Kruskal'Wallli  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Experiment  #1 
Experiment  #2 

4 

H«  3.5714 

P*  0.144 

7 

Sig."  JJg 

Sig.  NS 

Sprina.  197; 

Experiment  #1 
Experiment  #2 

 d... 

H"  i.20n 

p«  n.42fi 

Sig."  Nfi 

Sig."  Mfi 

Fall,  1970- 
Sprinp.  197: 

Experiment  #1 
Experiment  #1 

4 

H-  6.545 

,D"  0.008 

6 

Sig.-  .02 

Sig."  .01 

Experiment  #2 
xperiment  #2 

7 

H-  8.076 

p"  0.409 

 5__ 

Sig."  .01 

Sig.«  NS 
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QUESTIONNAIRE  DATA 


Item  I  have  good  ways  for  knowing  how  good  our  results 


are, 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5. .8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total 


Fall,  1970 


N 

-39 
60 

Ts 

164 


X 

4.97 


4.  45C 


5.13E 


SD 

1.202 


1.419 


1.073 


Spring,  1971 
— X2. 


N 

73 


50 


64 
187 


X 

4.876 


4.360 


4.796 


SD 

1.290 


1.289 
1.449 


Fall,  1971 


68 


45 


67 


241 


X 

4.48 


3.91 

TTTe 


SD 
1.151 


1.164 

rriT2 


T7m 


Spring,  1972 
-*4 


N 

40 

206 


X 

4.102 


4.050 


^7m 


1.218 
T7T24 


Two-Wav  Anal  of  Var. 

Tl    6  T2 

T2    g  T3 

T3  £ 

T4 

Tl    8  T4 

Experiment  SED#1  W/ 
Control  SED  #1 

F 

Si  gni  f 

p 

Signif . 

1.706 

NS 

1.687 

NS 

3.583 

NS 

i9.4fi7 

.001 

0.062 

NS 

0.355 

NS 

3.374 

NS 

MS  ' 

Experiment  SED#2  W/ 
Control  SED  #1 

1.591 

NS 

1.991 

NS 

0.185 

NS  .. 

.ni 

Row 

10.818 

.01 

14.046 

.001 

14.013 

.001 

10.542 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

3.489 

NS 

Row 

O.bbU 

N§ 

Col. 

0.196 

NS 

Row 

0.897 

MS-  ■ 
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QUESTIONNAIRE  DATA 


Item   s        My  organization  has  reliable  ways  fo:  knowing  how  well 
it  is  attaining  its  objectives.  


Scale 
Values 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
.4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
I.O 


Fall,  1971 

  T3 


Spring^^l972 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


N 

M 

45 
67 
TI 
2Ti 


X 

4.220 


3.600 
4.552 

TTm 


SD  ^ 
1.563 


1.23: 


1.282 


N 

39 
40 
61 

5o? 


X 

4.07611 


3.800 
09 


.30$ 


1.50$ 
7U 


Two-Wav  Analysis  o 

F  Var. 

Experiment  SED#1 

W/fontrnl  RPn#1 

"  1 

Signif, 

Cnl. 

US 

Experiment  SED#2 
W/Control  SPn#1 

rm 

Bow 

.001 

Experiment  SED#1 
If /Control  SPn#9 

£o.l. 

2.304. 

NS 

Row 

0.04-7 

NS 

Experiment  SED#2 
W/Control  SP.n#? 

Col. 

0.380 

NS 

Row, 

6.33.1 

.05 

QUESTIONNAIRE  DATA 


Item  9       I  think  that  the  objectives  developed  during  AMA 

training  are  clearly  stated  with  respecfc  to  results 


expecte 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.3 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Two  Way  Analysis  of 
Variance 


Experimental  SED#1 
hf/Experin.ental  SED#2 


Fall.  1971 


Spring.  1972 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.3 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Fall,  1971 
T3 

Spring,  1972 
T4 

Experimental  SED#1 

N 

68 

4.838 

SD 
1.767 

.  N 
39 

X 

5.205 

SD 
1.128 

Experimental  SED#2 

45 

4.177 

1.466 

39 

3.948 

1.571 

Total 

113 

78 

n1 


Esu. 


Signif. 


.QQl 


b03 


CONTEMT  ANALYSIS  DATA 

I tem    1  ;  Adequate  Resources  (money  and  information) 


Exp.«l 


H=  0.222:  Sig . »  NS 
p=  0.108;  Sig . »  NS 


Control  #1 


SCMEM/kTIC  PRESENTATION  OF  ANALYSIS 


Fall,  1970 


H«  0.044;Sig.»  NS 
P«  0.300; Sig . «  NS 


H=  0.170; Sig . »  NS 


P 0.005;Sig . »  .01 


Spring,  1972 


Exp.#l 


0 


H°  0. 179  ; Sig . =  NS 
p *  0.9  84  :Sig.  =  NS 


6 


Control  #1 


Fall,  1970 


Exp.«2 
H=  1.003:Sig.-  NS 


H«  n.n3iSig.«  NS 


p=-irTTTF:sig.-1IS- 


P=   0.22^ig.°  NS 


Control  #1 


H« 0.170  Sig.  =  NS 


P'O.OQS  Sig.»  .01 


Spring,  1972 


Exp. #2 


H«  0.283;  Sig .  =  NS 
p  =   0.182;  Sig .  =  NS 


Control  #1 


Eall-_ia20 


Spring.  1972 


Exp.  #1 


H»0.499  :Sig.»NS 
 ~ —  .ITS" 


:Sig.' 


Exp. #2 


H-0.044  ;Sig.' 


NS 


p=  0.  300  ;Sig.°  NS 


H»  0.031  ;Sig.' 


NS 


p«0.  224  ;Sig.a  NS 


-0 


Exp.n 


H=  0.070  :Sig.=  us, 

p  '  0.326  ; Sig .  =  NS 


6 


Exp. #2 


504 

It—    1  ;    Adequate  Resources  (money  and  information) 


Fall,  1970 

N 

Kruskal-Nallis  One-Nay 
Analysis  of  Variance 

Binominal  Test  of 
Proportions 

Experimentffl 
Control  #1 

9 

H-  0.222 

P  a    0.  108 

Signif.  =  NS 

6 

Signif.-  NS 

Experim6ntff2 

Control  #1 

9 

H«  1.003 

p  «  0. 108 
Signif. =NS 

6 

Signif.-  NS 

Experim6nt#l 
Experiment#2 

9 

H-  0.499 

0. 780 

9 

Signif.-  NS 

Signif.cNS 

Spring,  1972 

Experimenttl 
Control  #1 

7 

H-  0.179 

P  =  0.984 
Signif.=  NS 

? 

Signif.-  NS 

Experiment#2 
Control  #1 

6 

H»  0.281 

P=0.182 
Signif .=  NS 

9 

Signif.*  NS 

Experiment#l 
Exr8riment#2 

7 

H-  0.020 

P*0.326 
Signif.  =  NS 

6 

Signif.-  NS 

Fall,  1970- 
Spring,  1972 

Experiment  #1 
Experiment  #1 

9 

H>  0.044 

p=0. 300 

7 

Signif. -NS 

Signif.-  NS 

Experiment  #2 
Experiment  #2 

9 

H>  0.031 

p=  0.224 
Signify  NS 

6 

Signif.-  NS 

Control #1 
Control#l 

6 

H-  0.170 

p  =  0.005 

9 

Signif.-  NS 

Signif.  =  '01 
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OONTBMT  AHALYSIS  DATA 
Item  2    ;  Control  System  expressed  through  decision  making  process. 


Exp.fl 


H=  0.497;  Sig.  ■  NS 


p=  0.003  ;Sig. ■  .01 


Control  #1 


SCHEMATIC  PRESnfTATION  OP  ANALYSIS 
Fall,  1970  Spring.  1972 


H> 


H- 


P". 


_;Sig.. 


:Sig.> 


:Sig.> 


,;Sig.. 


0 


H"  0.284  : Sig."  NS 
p  *  0.778  :Sig."  NS 


Control  #1 


Exp.f2 


H=  3.093  ;Sl8.' 
p=  0.000  ;Sig.' 
 TTW 


Control  #1 


Fall,  1970 


H- 


NS 


,001 


H- 


Sig. 


Sig.> 


Sig.> 


Sig.* 


Spring,  1972 


Exp. #2 


H"1.852  ; Sig. «  NS 
p  =0.934  ;Sig."  NS 


Control  #1 


Fall,  1970 


Exp.  #1 


H- 


:Sig.> 


H"1.697   ; Sig. -  NS 


p<0.428   iSig. ■  NS 


:Sig.> 


Spring,  1972 
 0 


H'»0.409  : Sig . »  NS 
p  "0.532  ; Sig . "  NS 
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It—   2    :    Control  System  expressed  through  decision  making  process 


Pali,  1970 

N 

Kruskal-Wallis  One-W«y 

Anftlvslfl  o£  VariAnre 

Binominal  Test  of 
rroportions 

Experiment*! 
Control  #1 

10 

H-0.497 

P«  0.003 
Signif. =  .01 

6 

Signif.-  NS 

ExpGriment*2 
Control  #1 

Signif.*  NS 

P«  0.000490 

Signif. a  .001 

Experiment#l 
Experiment*;* 

10 

H-1.697 

P*  0.428 
Signif. »  NS 

11 

Signif.*  NS 

Spring,  1972 

Experiment* 1 
Control  #1 

4 

H*  n  5  Q  4 

6 

Signif.a  N<5 

Sisnif.s  NIC 

Experiment#2 
Control  #1 

6 

H»1.852 

P=0,934 

6 

Signif.*  NS 

Signif. =  NS 

Experiment#l 
Experiment#2 

4 

Ha  0,409 

0.532 

6 

Signif. «  NS 

Signif.  =  NS 

Fall,  1970- 
Spring,  1972 

Experiment  #1 
Experiment  #1 

10 

H«  0.9  80 

D=  0 . 008 

4 

Signif.- NS 

Signif.-  .01 

Experiment  #2 
Experiment  #2 

11 

H»  0.911 

px  0.006 

6 

Signif.-  NS 

Signify  .01 

ControX#l 
Control#l 

6 

H*  0.314 

P  =  0.274 

6 

Signif.-  NS 

Sxgnif.s  NS 
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QUESTIONNAIRE  DATA 


Itm    3      Mv  manager  makes  it  clear  that  he  is  committed  to  the  success 

of  nnr  nrn-i^r;t-g. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.3 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
.3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Fall,  1970 


N 
39 
60 
7S 

165 

X 

6.256 

SD 
0.938 

N 

73 

X 

6.260 

SD 
0.850 

6.100 

0.933 

52 
H 

4.846 

1.539 

57615 

1.667 

L86 

Spring,  1971 


Fall,  1971 

—ax 


N 

68 
45 
TT 

wnr 


X 


5.^ 
5.895 
5.^ 


SD 


N 

AH 

1.1291  61 


Spring,  1972 
-3:4- 


X 


4.69^ 


5.442 


5.53C 


SD 
1.641 


1.21  4 


1.1;  2 


TwQ-Wav  Anal  of  Var. 


Tl    6  T2 


T2    fi  T3 


T3  6 


Jl. 


.6  T4 


Experiment  SED#1  W/ 
Control  SED  »1 


13.847 


4.645 


Signify 


.001  I  0.816 


.05  111.622 


NS 
.001 


1.737 


■Signif 


fi.546 


3.967 


7.nfin 


gnif  ■ 


.01 


Experiment  SED#2  W/ 
Control  SED  #1 


2.133 


NS 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED* 2  W/ 
Control  SED  #2 


Row 


33.424 


5.930 


.05 


6.583 


.001  I  8.892 


.01 


13.813 


115  17.291 


.nni 


Col. 
Row 


0.000 
b.ib4 


■  OQl 


4.213 


.05 


NS 


Col. 


1.890 


Row 


8.134 


NS 

TTn: 
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QUESTIONNAIRE  DATA 


Item  4 


My  manager  has  expressed  the  belief  that  the  AMA's  training 
program  has  h^Pn  hi»iDf„i  .  2- 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

S.8 

S.6 

5.4 

5.2 

S.O 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED  #1 
Experimental  SED  #2 

Total 


N 

38 
60 

98 


Fall,  1970 
XL 

X  SO 
4.737L.996 


4.3672.099 


Spring,  1971 
— X2. 


N 


52 


X 


4.4.23 


SD 
l.?7? 


124 


Fall,  1971 


N 


15 


113 


X  SD 


4.4881.700 


Spring,  1972 


N 


79 


X     I  SD 
4.230  1.85 


Tl«tWav  Anal  of  Var. 

Tl  ^ 
p 

T2 

T2    «  T3 

T3  fi 

T4 

Tl 

6  T4 

Experimental  SED#1 
W/Experimental  SED#2 

4.938 

12-315 

Signif 
.05 

 F  |Siyn4f 

_a.i7Qj  ijs 

F 

0.010 

Signif 

_  US 

p 

2.fi3fl 

Signif 
NS 

.  .001 

J5.172I  .om 

32.943 

.001 

12.818 

.001 
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QUESTIONNAIRE  DATA 


Item    5        My  manager  understands  nlannina  theory  and  is  able 
to  put  it  into  practice.  


Scale 
Values 


7  n 

o  •  o 

O 

A  A. 
O  #*f 

A  9 

A  n 

D  •  O 

A 

t;  4 
J .  *f 

o  •  u 

*f  .  o 

d  A 

CI 

*f .  *f 

d  9 

*♦  #u 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall,  1971 

  T3 


Spring^^l972 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


N 

67 
1241 


X 

4,897 


4.940 


SD 


1.486 


N 

61 


206 


Jus 


i^oi^ii: 


SO 

an 


4.7211.34: 


^Two-lifav  Analysis^  a 

p  Var. 

)  T4 

Experiment  SED#1 
tt/CfiOlroI  SEDf 1  , , 

p 

rm 

0.050 

0.419 

.  m  . 

Experiment  SED#2 

2.S49, 

NS 

7.260 

.01 

Experiment  SED#1 

0.026 

NS 

3.596 

NS 

Experiment  SED#2 
W/Control  SED#2 

Col. 

1.533 

NS 

1.756 

NS 
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qUBSriONNAIR£  DATA 


I  believe  mv  organization  rrives  me  ar^A^uate  training 
to  do  mv  work  effeetivclv.  ^ 


Scale- 
Values 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 


4, 
4. 
4. 
3. 


3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 
T3 


Spring^^l972 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 

Total  


N 

67 


X 


4.880 
CIS 


SD 


1.  571 


1.  608 


N 

M 

61 


207 


X 


1^ 

4.  852 


4.651 


SD 
J^^4 


1.4: 


1.3^ 


Two-Wav  Analysis  o 

p  Var., 

Experiment  SED#1 

W/rontrnI  <?Pn#1 

.E,  .  , 

o.ono 

1.57,(1 

NS 

Experiment  SED#2 
W/Control  SFn#l 

rm 

NS 

..0. 671 

:is 

Experiment  SED#1 
W/Control  fiPn#5 

rni 

0.042 

4.411 

.05 

Experiment  SED#2 
W/Control  SPn#7 

Col. 

NS 

Row 

0.000 

NS 
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QUESTIONNAIRE  DATA 


Item  7 


I  feel  good  about,  mv  man^nf^r's  ahn,4-v  t-n  p];,n 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Fall,  1971 

T3 


Spring^^l972 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 


N 

67 
Tl 


5. 


21 

n 


SD 


l.SRfi 
1.222 


U^7 


,  Two-Wav  Analysis  o 

f  Var. 

 li.„| 

i  T4 

Experiment  SED#1 

WVControl  «?Pn#l 

NS 

Experiment  SED#2 
W/Control  SPn#l 

rm 

NS 

LRqu-. 

19.nQi 

.001 

Experiment  SED#1 

rm 

0.688 

NS 

:.424. 

N5 

Experiment  SED#2 
W/Control  «5Rn#2 

Col. 

NS 

Row 

8.  887 

.01 

N 

ASL 
21. 
il 

06 


X 


5.^2if 


5.J22P 


SD 
J.JJI1 


1^9 
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qUBSTIONNAIRE  DATA 


My  manager  provides  me  with  adequate  support  to 
perform  ray  job. 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

S.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


X 


Fall,  1971 
T3 


Spring_j^^l972 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


N 

67 
k41 


X 


5.641 


SD 


1.21^ 


N 

60 
66 


X 

5.47fi 


5.183 
L^7 


SD  ^ 
1.24p 

ITTB3 


^Tw9-w^Y. Analysis  a. 

F  Var. 

F.  TA 

Experiment  SED#1 
W/Control  SF.n#l 

0.999 

NS 

Experiment  SED#2 

W/rontrol  SFn#l 

roi 

7.781 

.01 

12.240 

.001 

Experiment  SED#1 
W/Control  SPn#^ 

nm. 

2.201 

NS 

Rnw 

1.019 

NS 

Experiment  SED#2 
W/Control  SEDf2 

Col. 

5.475 

.  05 

Row  . 

11.621- 

.001 
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QOWTBWT  iUULYSIS  DATA 
I  tern  1  ;   Establish  credibility  of  Pj.c^nni,nq. 


Fall,  1970 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Spring,  1972 


Exp.  ifl 


H=  1.  208  : Sig. "  NS 
p=  0.736  ; Sig. •  NS 


Exp. #2 


6_ 


H'O.IOO  ;Sig.  NS 


D'^Q.onn  :Sig;  .nm 
001 


H"8.040  ; Sig . ■  .01 


p  a0.104  ; Sig. «  NS 


Exp.*l 


'O 


H°  8.a98:Sig.='  .m 
P=  0.000:  Sig .  =  NS 


Exp. #2 


Fall,  1970 

N 

Kruskal'Wallii  One-Way 

Analysis  of  Variance 

Binomial  Test  of 

PT«pnTtir>n«: 

Experiment  #1 
Experiment  #2 

7 
8 

1.208. 
Sig." 

Sig.  NS 

SDrina.  197: 

Experiment  #1 
Experiment  #2 

9 
8 

H-8.898 
Sig.-.  01 

p ■ 0. 000 
Sig.»  NS 

Pall,  1970- 

Spring,  197; 

Experiment  #1 
Experiment  #1 

7 
9 

H«  0.100 
Sig.>  NS 

-o»  0.000001 
Sig.-  .001 

Experiment  #2 
xperiment  #2 

8 
S 

H-8.040 
Sig.-  .01 

p-  0.104 
Sig.  -  NS 
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OONTEMT  MkUSlS  DATA 

I tem  2     ;  Role  of  Planning  -  How  integral. 


SCHEmTIC  PRESMfTATIOH  OP  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.«l 


H=0.067  : Sig. ■  NS 
p=O.OQO  -Sig..  NS 


Control  #1 


H"  0.505  ;Sig..  NS 


P*  0.000  : Sig . "  NS 


H«  7.841  ; Sig . ■  .01 


p ■  0.000  ;Sig . ■  NS 


Exp.#l 


H«  7.689; Sig .  =  .01 
p  *  0.000; Sig .  =  NS 


Control  #1 


Fall,  1970 


Exp.«2 

H=5.333  ; Sig. -  .05 
p=  0.000  ; Sig . ■  NS 


Control  #1 


H»  0.004 Sig . »  NS 


P ■  0.  OOQSig . »  NS 


H- 


Sig.= 


Sig.* 


Spring,  1972 


Exp. #2 


H''  1.042;  Sig.  =  NS 
P  =  0.000;Sig.=  NS 


Control  #1 


Exp.  #1 


Fall,  1970 


H«.500    ;  Sig . "  .01 


P»n.nnn  :Sig."  ns_ 


Exp.*2 


H»    0.503Sig. -  NS 


P*  Q.QQQ:Sig.»  NS 


""JUUU.;Sig.»_iis_ 


P *  0.000  ;$ig. »  NS 


Spring,  1972 


Exp.#l 


H=  4.506;sig.»  .05 
p  s  0.000; Sig . a  NS 


Exp. #2 


.ERIC 
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It«a 


.*     Role  of  Planning  -  Hr>M  integral, 


ERIC 
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CONTBIIT  AMALYSIS  DATA 

Item  3    :     Role  of  Planning  -  How  much  is  needed. 


Exp.fl 


H=  1.  470: Sig . «  ns 
p-  O.OOQ: Sig. »  NS 


Control  #1 


SCHEMATir  PBRgiihffATION  OP  ANALYSIS 

Fall,  1970  Spring,  1972 


H» 0.247  ;Sig.«  NS 


p» 0.000  ; Slg . »  NS 


H-J^lfii_:Sig.-  NS. 


p .0.000  ; Sig . »  NS 


Exp.#l 


0 


H«  Q.n7n  ;Sig.=  ns 
p *  conn  : Sig . ~  ns 


Control  #1 


ERJC 


Fall,  1970  Spring,  1972 

Exp.  #2  H-^i^i^Sig.  ^ 


H=  3.  203;Sig.»  NS 


P=  O.OOQ:  Sig . «  NS 


Control  #1 


P'Q.oon  Sig.= 


H-2.181  Sig.  NS 


E3<p.#2 


H"  0.150  : Sig .  =  NS 
P  =  n-nnn;Sig.= 


p 'n.nnn  Sig . *  ns 


Control  #1 


Fflll,  1970 


Exp.  #1 


247:Sig.a  NS 


H» 0 . 163  ; Sig. »  NS 


p*0.000  :Sig . "  NS 


p'    n.nnn:Si£..  jj^. 


Snriny.  1Q77 

 0  ^^-^^ 


H-  0.792;Sig.: 


NS 


0.481  :  Si^ .  =  NS 

p  =  n.nnn  ♦sig-=  ns_ 


Exp. #2 


p "  0.000; Sig . «  NS 


Exp. #2 
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It—    3  :     Role  of  Planning  -  How  much  is  needed. 


Pall,  1970 

N 

Kruskal-Nallis  One-Way 
Analysis  of  Variance 

Binominal  Test  of 
Proportions 

Experimentfl 
Control  #1 

.12 

H-  1.470 

P  a  0.000 

Signif.*  NS 

12 

Signif.>  NS 

Experiiidntff2 
Control  #1 

12 

H»  3.203 

p  «  0.  000 

12 

Signif.-  NS 

Signif.  =  NS 

Experiment#l 
ExperiDent*2 

12 

H»  0.163 

p»0.000 

12 

Signif.*  NS 

Signif."  NS 

- 

- 

Spring,  1972 

Experiment#l 
Control  #1 

9 

H-  0.070 

P+0.000 

11 

Signif."  NS 

Signifj«NS 

Experiment#2 
Control  #1 

10. 

H«  0.150 

P=0.000 

11 

Signif.  «•  NS 

Signif.  «NS 

Experiment#l 
Experiment #2 

9 

H»  0.481 

P*0.000 

10 

Signif.*  NS 

Signif.  =  NS 

Fall,  1970- 
Spring,  1972 

Experiment  #1 
Experiment  #1 

12 

H»  0.247 

p=0. 000 

9 

Signif.*  NS 

Signif."  NS 

Experiment  #2 
Experiment  #2 

12 

H-  0.792 

p=  0.000 

10 

Signif.-  NS 

Signif?  NS 

Controlfl 

9, 

Control#l 

1? 

H«  2.181 

p  s  0.000 

11 

Signif.-  NS 

Signif.-  NS 

5ie 


ooNTarr  ahalysis  data 

I tem  4    :     Role  of  Planning  -  emergence.   


Exp.n 


Control  #1 


SCHEMATIC  PRESPffATIOH  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


H=  1.687- Sig.  ■  NS 
p=  0.000;  Sig .  ■  NS 


H«3.681  ;  Sig.  ■  ■  NS 


P" 0.000  ; Sig . ■  NS 


H«8.015  ; Sig. ■  .01 


P" 0.000  ;Sig. ■  NS 


Exp.#l 


o 


H"10. 181  ;Sig.g  .01 
p ♦  0.  000  :Sig.g  NS 


Control  #1 


Fall.  1970 


Exp. #2 

H=  3.967; Sig .  ■  .05 
0.000; Sig . ■  NS 


Control  #1 


H«3.531  Sig.' 


NS 


P "0.000  Sig . «  NS 


H'8.015  Sig.=  .01 


P '0.000  Sig. ♦  NS 


Spring,  1972 


NS 


H=  0.243  ;Sig. 
P  =  0.000  ;Sig.=  NS 


Control  #1 


ERIC 


Exp.  #1 


Fall.  1970 


H»2.167  : Sig. "  N^ 


D*0.000  ; Sig . ■  NS 


Exp.«2 


He  3.681;Sig.. 


NS 


P  *    0.000:  Sig .  »_NS 


H-  3^531; Sig. « 


NS 


P  ■  o.ooo;  Sig .  »_NS 


Spring,  ;372 


Exp.#l 


H:d.l.760  ; Sig .  =  .001 
P  =  0.000  ;Sig.-  NS 


Exp. #2 
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It—    4  ;  Role. of  Planning  -  emergence. 


ERIC 


Fall,  1970 

N 

Kruskal-Nallis  One-Nay 
Anal/sit  of  Variance 

Binominal  Test  of 
Proportions 

,  Exporimentfl 

12 

H-  1.687 

P  »  0.000 

Control  fi 

12 

Signif.>  NS 

Signif. »  NS 

Experlaent#2 
Control  #1 

12 

H-  3.967 

p«  0.000 

12 

Siffiif.*  .05 

Signif.  =  NS 

Experimental 
Experiment#2 

12 

H»  2.167 

p«  0.000 

12 

Slgnif.a  NS 

Signif."  NS 

Spring,  1972 

Experiment* 1 
Control  #1 

H-  10.181 

P==0.000 

Slgnif.*  ,01 

Sigr,.x.»  NS 

Experiment#2 
Control  #1 

10 

H-  0.243 

P=0.000 

11 

Signif.*  NS 

Signif.-  NS 

Experiment #1 
Experiment#2 

9 

H-  11.760 

P*0.000 

10 

Signif.-  .001 

Signif.-  NS 

Fall,  1970- 
Spring,  1972 

Experiment  #1 
4 

Experiment  #1 

12 

H>  3.681 

D«0. 000 

9 

Signif.  .NS 

Signif.- NS 

Bxperlnont  #2 
Experiment  #2 

12 

H«  3.531 

A  AAA 

p«  0.000 

10 

Signif.-  NS 

Signif*  NS 

Controlil 
Control #1 

12 

Hm  8.015 

p  •  0.000 

11 

Signif.*  .01 

Signif.-  NS 
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QUESTIONNAIRE  DATA 


IteiQ__5  As  I  see  it,  planning  is  an  integral  part  of  running  the 


SCALE 

VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
S.8 
5.6 
5.4 
5.2 
.  5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Contro    SED  #1 
Control  SED  #2 
Total  


Fall,  1970 

Tl 

N 

39 
60 

16f 

X 

6.564 

SD 
0.718 

6.533 

0.853 

6.500 

0.662 

Spring,  1971 
—12- 


N 

73 


X 

16.342 


52, 
61 


86 


5.712 
TDT 


SD 
0.961 


1.661 


Fall,  1971 


N    j  X 
68  k.691 
45  k.066 
6TkT40^ 


SD 
0.653 


N 

40 

l.llj  40 

OsslTl 
IT 


7  n 

f%  ft 
o « o 

w .  O 

Q  •  U 

^  •  o 

^  fi 
^  •  o 

<;  2 

o  •  u 

d  ft 
*♦  •  o 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring,  1972 
-14. 


X 

5.9251 


5.fi!iC 


SD 
_1_.JL35 


Two-Wav  Anal  of  Var. 

Tl  6 

T2 

T2 

S  T3 

T3  \ 

T4 

6  T4 

P 

£i«nif. 

F 

Signif 

-  P 

Experiment  SED#1  W/ 
Control  SED  #1 

low 

1.156 

NS 
.05 

.01 

19.835 

.001 

15.307 

.Signif  

.001 

Experiment  SED#2  W/ 

;oi. 

1.650 

MS 

-4. SOI 

3.529 

.  .03 

-  H? 

0.22£ 
6.79- 

NS 
.01 

.147 
23.045 

NS 
.001 

Control  SED  #1 

>ow 

15.679 

.001 

4.998 

.05 

7.88i 

.01 

3.208 

NS 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

17.645 

.001 

Row 

2.51t 

NS 

Col. 

6.639 

.01 

Row 

5.75^ 

nS 
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QUESTIONNAIRE  DATA 


Item  6 


As  I  see  it,  persons  in  this  organization  put  a  lot  of 
effort  into  planning. 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

Z.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total  


Fall,  1971 

  T3 


N 

M 

§1 

61 


X 

5.147 


4.??? 
4.880 
4.934 


SD  ^ 


1x355 


1.40S 

TTsTj 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

c» 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring^^l972 


N 

M 

60 

H 

206 


X 

4.62511 


4.5501 

TTTTgi" 


.211 


4j.075ll.36a 


.28: 


1  Two-Wav  Analvsis  o 

P.  Til 

Experiment  SED#1 

W/ron+rn1  RRn#1 

rm, 

fi.m5 

.05 

NS 

Experiment  SED#2 
tf/Controi  sp.n#i 

q.77Q 

.01 

^-f 

.  .1.232. 

NS 

Experiment  SED#1 
W/Control  SRn#9 

rm 

8.30? 

.01 

1.273 

NS 

Experiment  SED#2 
W/Control  SRn#? 

.Col. 

12.289 

.001 

LRow 

0.846 
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QUESTIONNAIRE  DATA 


Item  7 


My  capability  to  plan  effectively  will  positively  affect 
my  future  career  in  this  nrgani  7.afir»n .  


Scale 
Values 


7.0 
6.8 


6 
6 
6 
6 
5 
5 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
•2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 
T3 


3pring^^l972 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 

Total  


N 

67 

H 

241 


X 

5.764 


5.555 


5.098 


SD 


N 

39 


1.815 


1.317 

rreTo 


39. 
61 


205 


X 

5.461 


4.564 
5.032 

5TTn"5 


1.09^6 


1.5 


1.  5C  5 


Two-Wav  Analvsis  o 

e  \I»r. 

Experiment  SED#1 

W/rontrol  SP.n#1 

F 

rnl , 

5.212. 

.tiS 

3.167 

N§ 

Experiment  SED#2 
W/Control  SPn#1 

rnl 

.  5.112 

.05 

Hnw 

5.331 

.05 

Experiment  SED#1 
w/roni-roi  spn#:> 

rnl 

1.06Q.. 

NS 

Row 

8.796 

.01 

Experiment  SED#2 
W/Control  SFn#2 

lol. 

1.460 

NS 

Row 

1.385 

NS 

ERIC 
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QUESTIONNAIRE  DATA 


Item  8        The  activities  relating  to  planning  are  having  an 
effect  on  the  policy  of  this  organization. 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

e» 

4.8 

c« 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall,  1971 
T3 


ei 


Spring_j^^l972 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

l.O 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 
Total 


N 

67 


X 


4.850 
4x222 


SD 


N 

AO. 

1.4161  59 


203 


X 


4.694 
1^4 


SD  , 
1.141 


1.421 


1.392 
1.297 


Two-Wav  Analysis  o 

P  Var. 

Experiment  SED#1 

W/Contrnl  SPn#1 

p 

n.n45 

.01 

Experiment  SED#2 
W/Control  SPn#1 

rm 

5.087 

.05 

JEinw 

0.0724 

NS 

Experiment  SED#1 
W/Control  SPn#5 

0.207 

NS 

Row 

11.943 

.001 

Experiment  SED#2 
W/Control  SF.D#2 

Col. 

6.264 

.05 

Ran,.  , 

o.i§§ 

NS 

QUESTIONHAIRE  DATA 


-  ''   ""^^  ^-7-- ration  ar.  ....^..n,.^^ 


SCALE 
VALUES 


7 
6. 
6.6 
6 
6.2 
6.0 
5 
S.6 
5.4 
S 

s.o 

4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


St 


lixperiment 
l.xperiment 
l.xperiinent 
lixperiment 
Control  LEA 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experiment  LEA#4  W/ 
Control  LEA  #2 
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QUESTIONHAIRE  DATA 


Item  2 


:  Our  goals  are  realistic  and  attainable  with  our  best  efforts. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall  '70  T, 


Spring  '71  T 


Fall  '71  T, 


T 

32 
31 


7.0 

6.8. 

6.6 

6.4 

6.2 

6.0 

ei 

5.8 

5.6 

5.4 

C  L 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring  '72 


N 

X 

SD 

N 

X 

SD 

hxperiment  LEA  #1 

28 

5.392 

0.  831 

38 

5.289 

1.183 

l.xperiment  LEA  #2 

25 

4.  920 

1.605 

26 

4.769 

1.242 

l.xperiment  LEA  #3 

33 

4.878 

1.386 

17 

4.8:^3 

1.333 

Experiment  LEA  #4 

26. 

4.  846 

1.317 

?J 

4.565 

2.128 

Control  LCA  #1 

Control  LEA  #2 

Total 

112 

10< 

Two- Way  Anal.ofVar. 


T2  S  T3 

~T  


ii 

29 
30 
36 
182 


T 

2  IBID 


9353.928 


666  J. 816 


SD 

1.750 


31 


32 


23 


2411.745 


2001.761 


27 

To 


6381.198 


33 


166 


032 


625 


434 


074 


850 


333 


SD 
0.7915 


1.099 


0.99  2 


1.567 

"OTSS 


1.020 


Tl  §  T2 
T 


Signif . 


T3  S 
T — 


T4 


Tl  S 

T — 


T4  

Signif. 


fSxperiment  LEA#1 
iixperiment  LEA#2 
Rxperiment  LEA#3 
Experiment  LEA#4 


W/Col 
R 

¥7S 


Row 


4.640 


0.303 


.05 


32.740 


NS 


1.015 


.001 


2.376 


NS 


4.588 


Signif, 


■NS 


7-4'^4 


05 


2.044 


NS 


01 


Row 


0.269 


NS 


5.032 


.05 


0.540 


NS 


2.  270 


0.201 


NS 


1.018 


NS 


2.097 


JiS. 


0.571 


Experiment  LEA#1  W/ 
Control  LEA  #1 
Experiment  LEA#2  W/ 
Control  LEA  #1 


Col 


2.976 


NS 


Row 


Col 


0.417 


NS 


Row 


Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  W/ 
Control  LEA  #2 


Col 
Row 


1.801 


NS 


NS 


U.1U8 


NS 
TIS" 


Row 


JIS- 


1.729 


NS 
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QUeSTIONHAIR£  DATA 

The  ton  priority  obiectiyey  for  state  education  are 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

S.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Fall,  1971 


Spring,  1972 
X 

5.161 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2. 

4.0 

3.8 

3.6 

3.4 

3.2 

3,0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


N 

32 
31 

3£ 
82 


X 

5.250 


4.870 


SD 

i.2ia 


1.38^ 
OTf 


N 

.31 

32 

166 


3.. 15^ 


Two-Wav  Analysis  oi 

par. 

T3    fi  T4 

Experiment  LEA#1 
W/ExDeriment  LEA#2 

"  ] 

Signif. 

Col. 

0.084 

^ow 

0.??7., 

NS 

Experiment  LEA#3 
V/Experiment  LEA#4 

Col. 

0.647 

NS 

How 

0.803 

NS 

Experiment  LEA#1 
W/Control  LEA#1 

Col. 

0.000 

NS 

Row 

0.040 

N^ 

Experiment  LEA#2 
IZControl  LEA#1 

Ot4?.l, 

NS 

Row 

1.186 

N^ 

Experiment  LEA#5 
W/Control  LEA#2 

Col. 

0.115 

NS 

Row 

6.614 

■    NS  ■  ■' 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

0.101 

NS 

Row 

0.745 

"■'MS'  ' 

5.093 

Oob 


3T7T4 


5,3QP 


4.363 


SD 


1.4 


l.K 

T7?( 


8 


8 


1^ 


8 
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QUESTIONNAIRE  DATA 


Item    4      I  feel  that  the  objectives  developed  during  AMA  train- 
ing reflect  the  most  serious  and  pressing  needs  of 


state  education. 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

•2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


X 


61 


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

X 

SD 

Experimental  LEA  #1 

.?2 

5.843 

0-919 

31 

5.806 

1.077 

Experimental  LEA  #2 

31 

5.612 

1. 

406 

32 

5.593 

1.240 

Experimental  LEA  #3 

24 

4.458 

1. 

744 

5.000 

1.493 

Experimental  LEA  #4  . 

29 

4.931 

1. 

579 

2$ 

5.076 

1.622 

Total 

IW 

III 

Two-Way  Analysis  of 
Variance 

T3    S  ' 

^4 

F 

Signif . 

Experimental  LEA#1  W/ 

C9l. 

0.018 

NS 

Experimental  I,RA#2 

Row 

1.123 

NS 

Experimental  LEA#3  W/ 

C9li 

1.120 

NS 

Experimental  LEA#4 

Row 

0.715 

NS 

el 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 
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QUESTIONNAIRE  DATA 


As  I  see  it»  the  operational  priorities  of  the 
objectives  developed  during  AMA  training  are  clear. 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

■2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


X 


Fall,  1971 
T3 


Spring.  1972 


7 
6 
6 
6 
6 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 

Spring,  1972 

T3 

T4 

N 

X 

SD 

N 

X 

SD 

Experimental  LEA  #1 

.32 

n.fi79 

31 

1.067 

Experimental  LEA  #2 

31 

5 

.483 

1.338 

32 

5.531 

1.319 

Experimental  LEA  #3 

1 .791 

—21 

1.283 

Experimental  LEA  #4 

29 

4 

.R71 

1  .fi91 

Total 

]lfi 

Two-Way  Analysis  of 
Variance 


Experimental  LEA#1  W/ 
Experimental  I.EA#2 

F 

Signif . 

Cel. 

0.075 

NS 

Row 

3.927 

.05 

Experimental  LEA#3  W/ 
Experimental  LEA#4 

£&1. 

0.241 

NS 

Row 

0.010 

,  MS 
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QUESTIONHAXRE  DATA 


Item  _6  ;  The  kinds  of  things  I  am  doing  will  make  a  long  term 

contribution  to  education* 


ERIC 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


hxpcriment  LEA  #1 
lixperiment  LEA  #2 
l:xperiinent  LEA  #3 
l-.xperiment  LEA  #4 
Control  LEA  #1 
Control  I.EA  #2 
Total 


Two- Way  Anal.ofVar 


fixperimrnt  LEA#1  W/|CoI 
[.xperiment  LEA#2  |Row 


l;xperiment  LEA#3 
hxpcriment  LEA#4 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Experiment  LEA#4  W/ 
Control  LEA  #2 


Fall  '70  T, 


N 

X 

SD 

28 

5. 

821 

0.772 

39 

5. 

256 

1.25C 

33 

4. 

939 

1.17C 

27. 

5. 

444 

1.57- 

127 

Tl  5  T2 


Spring  '71  T. 


N 
25 
T6 


17 
23 


2l 


w7coi 


Row 


4.298 


1.381 


1.045 


0.053 


Signif . 


_,J15^ 


NS 


NS 


NS 


X 

5.040 
37^8 
5.058 
4.695 


T25T3 


SD 
1.670 

rrm 

1.197 
1.940 


I Signif 


10.96t  .OOT 


0.53a  NS~ 


2.02 


i 


NS 


0.091  NS 


Gol 


Row 


Col 


Row 


Col 
Row 


Col 


Row 


Fall  '71  T, 


32 


_2.4 
_2_9 
^0 
_i6 


i.937 
77^2 


5 

5^ 
5.T50 
5.413 
5* 
5' 


i.366 
i.416 


T3  § 


SD 
0.715 


7.0 

6.8  - 

6*6 

6*4 

6*2 

6*0 

5*8 

5*6 

5*4 

5*2 

5*0 

4*8 

4.6 
4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring  '72  T4 


0.882 


1.35S.-2.. 


1.12s 


0.431 


0.074 


0.  202 


0.781 


0.829 


3.388 


2.238 


0.583 
0.538 


0.202 


0.066 


4_  

Signif, 


NS 


JiS_ 


JiS_ 


NS 


NS 


NS 


NS 


NS 


N 
_31 

-1.2 
23 
27 
20 
33 
167 


IT 

5.  548 


SD 
0.99H 


5.781  1.12g 
5.043  I.39I7 


5.370  1.445 


5.4001^73 


5.242  1.173 


Tl  S  T4 


0.769 


n.003 


2.411 


Signif. 


NS 


NS 


NS 


NS 


Item 
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QUESTIONHAIRE  DATA 


As  I  see  it.  mv  nrganizat.inn  is  mt^ving  in  thft  yjght  direction. 


ERIC 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
S.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Two-Wav  Analysis  oi 

I  Yar. 

T3    fi  T4 

Experim'^nt  LEA#1 
i/Exoeriment  LEA#2 

P 

.  Signify 

Col. 

0.000 

NS 

Kow  1 

.^..•549 

NS 

Experiment  LEA#3 
W/Experiment  LEA#4 

Col, 

NS 

Row 

3.859 

NS 

ExperLnent  LEA#i 
tf /Control  LF.A#1 

Col. 

0.068 

NS 

Rov 

1 . 34s 

NS 

Experiment  LEA#2 
in/Control  LEA#1 

9.118  , 

m 

Row 

0.420 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

1.350 

NS 

Row 

0.620 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

i.m  . 

_  MS  ,  . 

Row 

1.191 

NS 

FalL 


N 

12- 
2Ji_ 


1971 
13 


X 


125 


0^ 


m 


SD 


1.564 
O.gSP 
1.42^ 


Spring,  1972 


N 

IL. 

21. 
22- 


X 


5.967 


S.22? 


ft.l.'iP 


5.M6 


SD 
n  9^1 


7.0 
6.8 
6.6 
6.4 
6.'' 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 
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QUESTIONNAIRE  DATA 


Item     1  :        organization's  policy  statements  are  clear. 


SCALE 
VALUES 


lixperiment  LEA  #1 
iixperiment  LEA  #2 
Lxperin:^nt  LEA  #3 
Lxpcriment  LEA  #4 
Control  LE-A  ^1 
('/-itrol  LEA  #2 

Total  

Two-Way  Anal.ofVar 


lixperiment  LEA#1 
I:X])eriment  LEA#2 


Hxperiment  LtA#i 
lixperiment  LEA#4 


lixperiment  LEA#1  W/ 
Control  LEA  f^l 
Experiment  LEA#2  W/ 
Control  LEA  #1 
Experiment  LEA#3  W/ 
Control  LEA  #2 
Experimjnt  LEA#4  W/ 
Cv'^ntrol  LEA  #2 
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QUESTIONNAIRE  DATA 


Item    2    :  My  organization's  performance  standards  are  understood. 


SCALE 
VALUES 


6.0 
S.8 
S.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall  '70  T, 


Spring  '71  T, 


Fall  '71  T, 


Spring  '72 


7.0 

6.8. 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


hxperiment  LEA  #1 
I'.xperiment  LEA  #2 
l.xperiment  LEA  #3 
Hxperiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Two-Way  Anal.ofVar 


N 
28 
39 
33 
57 


127 


X 

5.  464 
5.256 
5. 181 


SD 

0.  881 
1.312 

1.  210 
1.677 


N 

25. 
IL 
12 
21 


4.3fi0 

4.470 
4.52; 


SD 

1.776 

1.38fi 

1.903 


T 

-12 

-J.1 
-24 
-29 
_10 


X   I  So 

5.281 n.QQI 

4^01.478 
4.54in.q^1 
4.2751 

4x1241 


TT 


31 


23 


i.ain 
ill 


27 


20 


1.214 


32 


167 


4.521 


3.925 


4.650 
4.031 


i.ools 

1.174 


1.038 


1.639 


1.268 


1.307 


Tl  %  T2 


Signif. 


T2  8  T3 


Signif 


T3  S  T4 


Signif. 


Tl  S  T4 


Signif. 


I-xperiment  LEA#1  W/  Col 
tixperiment  LEA#2  (Row 


Experiment  LEAffS  WVCol. 
lixperiment  LEA#4  IRow 


18.23 


001 


6.  045 


0.547^  NS 


2.  657 


.  05 


0.649 


NS 


11.662 


8.95Q 


.001 


5.655 


JO. 


2. IK 


NS 


0.068 


NS 


0.420 


0.37( 


NS 


0.102 


7.298 


.01 


NS 


2.284 


J1S_ 


3.607 


JJS- 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Gol 


0.422 


NS 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Row 


4.431 


05 


Col 


0.430 


NS 


Row 


0.848 


NS 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Experiment  LEA#4  W/ 
Control  LEA  #2 


Col 
Row 


0.174 


NS 


3.643 


Col 


Row 


J1S_ 


0.001 


NS 
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QUESTIONNAIRE  DATA 


Item   :  Good  ways  are  used  to  let  me  know  how  I  can  improve  my 

performance,  


SCALE 
VALUES 


hxperiment  LEA  #1 
lixperiment  LEA  #2 
hxperiment  LEA  #3 
lixperiment  LEA  #4 
Control  LEA  #1 
Control  LI: A  #2 
Total 


IVo-Way  Anal.ofVar 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

e«- 

4.6 

e* 

4.4 

4.2 

ej 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall  '70  T, 


N 
28 
39 
33 
27 


127 


X 

4.714 
4.769 
4.242 

org 


SD 
1.  301 
1.404 
1.323 
1.875 


Tl  S  T2 


Spring  '71  T 


N 

25 
26 

17 
23 


91 


X 

4.  477 
4.  269 
4.235 
4.  304 


T2  S  T3 


SD 
554 
1. 115 
1.521 
1.  987 


Fall  '71  T, 


32 


li 
24 

29 


30 
36 
185 


SD 
0.973 


1.44E 


0.99 


1.532 


1.436 


1.61C 


T3  5 


T4 
Signif . 


Spring  '72 

r- 


31 


32 


23 


27 


20 


33 


167 


.696 


LJ. 


1^; 


Tl  5 


T4  

Signif. 


Experiment  LEA#1  W/jCol 
lixperiment  LEA#2 


Row 


Experiment  LEA#3 
Experiment  LEA#4 


Row 


5.731 


0.  408 


0.046 


0.  001 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Signif. 


.05 


NS 


NS 


23.03a  .001 
0.64j  NS 
4.021  .05 


0.075 


5.474 


NS 


0.2Bf|  NS 


3.116 


Gol 


2.319 


0.  186 


Row 


0.970 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Col 


1.231 


Row 


1.004 


Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  W/ 
Control  LEA  #2 


Col 


Row 


0.917 


Col 


Row 


1.031 


NS 


5.375 


,05 


0.  307 


_NS 


0.426 


NS 


0.909 


,01 


NS 


NS 


NS 


NS 


NS 


NS 


NS 


NS 


_NS_ 


NS 
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QUESTIONNAIRE  DATA 

I  understand  what  results  must  be  produced  to  achieve 
the  stated  objectives  of  this  organization. 


SCALE 
VALUES 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total  

Two-Wav  Analysis  of  V^ir 


Experiment  LEA#1 
W/Experiment  LEA#2 

Experiment  LEA#3 
W/Experiment  LEA#4 

Experiment  LEA#1 
W/Control  T.FA#1 


Experiment  LEA#2 

Itf/rnTitrnI  LEA#1 


Experiment  LEA#3 
W/Control  LEA#2 

Experiment  LEA#4 
W/Control  LEA#2 


Item 
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QUESTIONNAIRE  DATA 

The  Planning  nrii-  has  h^^^n  h«a1pfnl  n.o 


ERIC 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total  . 


Two -Way  Analysis  o 


Experiment  LEA#1 
W/Experiment  LEA#2 

Experiment  LEA#3 
W/Experiment  LEA#4 


Experiment  LEA#1 
ly/fnntrol  LEA#1 


Experiment  LEA#2 
1^/(^f>ntro1  LRA#1 


Experiment  LEA#3 
W/Control  LEA#2 


Experiment  LEA#4 
W/Control  LEA#2 


Col, 


Col, 


Row 


Col, 


2m. 


CoL 


Row 


Col. 


Row 


Col. 


Row 


Fall,  1971 
T3 


N 
32 

sT 

24 

K 
IQ. 

1821 


X 

5.843 


5.516 


5.166 


5.310 


5.066 
4.222 


SD 
0.954 


1.338 


1-659 


1.583 


2.099 


T3    8  T4 


1.1Q4 


,3.g77 


1.Q39 


0.093 


0.  362 


3.939 


0.060 


0.  092 


0.243 


9.  323 


1.528 


8.  285 


NS 


05 


NS 


NS 


NS 


05 


NS 


NS 


NS 
ITT 


.01 


Spring,  1972 
T4 


N 

21. 

32 
22 
2.5. 
20 


5.218 


27 


166 


5.090 


5.500 
3.962 


SD 

iaJLOie 


1.263 


1.269 
1.535 


1.357 


1.6C4 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 
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QUESTIONNAIRE  DATA 


I  ten  6 


I  have  good  wavs  for  knowing  how  gr.nri  r.u.  >-^sults  are 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall  '70  T, 


l.xperiment  LEA  #1 
l-.xperiment  LEA  #2 
l.xperiment  LEA  #3 
l-.xperiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Two-Way  Anal.ofVar 


lixperiraent  LEA#1  W/  Col 
Lxperiment  LEA#2  [Row 


Experiment  LEA#3  W/ Col 
Experiment  LEA#4  [Row 


N 

28. 
39 

|T|4 
27 


12  7 


X 

857 
641 

HE 
4.  851 


SD 
1.044 
1.307 

1.616 


Tl  §  T2 


Spring  '71  T, 


N 
25 


26 
IT 
23 


91 


1.650 
0.347 


0.041 


0.248 


Signif. 


JiS. 


NS 


NS 


4.480 
4.423 
TrgT7 
4.782 


T2  S  T3 


SD 
1.475 
1.064 

oTSn 

1.881 


8.616 


3.715 


0.814 
0.000 


Signif 


NS 


NS 


NS 


Fall  '71  T, 


32 
31 
T4 


29 
"To 

36 
"95 


SD 
0.983 


1.295 


31 
32 
23 
27 


0.  806 


1.371 


1.151 


T3  S  T4 


0.610 


11.044 


1.121 


0.360 


Signif 


NS 


001 


NS 


NS 


Spring  '72  T. 


33 
67 


T  ™ 

5.193|0.98Cl 


4.75010.982 
477§20.95] 


5.0001 


4.4541 


T1JT4 


4.6291. 62( 


1.306 


2.865 


0.070 


0.000 


Signif. 


NS 


NS 


NS 


7.0 
6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Gol 


0.100 


NS 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Row 


8.617 


Col 


1.326 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Row 


0.027 


Experiment  LEA#4  W/ 
Control  LEA  #2 


Col 
TtovT 


0.998 


^oT 


4.105 


Row 


Q.716 


1.091 


01 


NS 


NS 


NS 


05 


NS 
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Item 


QUESTIONNAIRE  DATA 

MY  nrflflnl7fit;i<7n  hn^  rpliflhlp  wavs  for  knowing  hnw 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 


SCALE 
VALUES 


5, 
4, 
4. 
4. 
4. 


4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Fall,  1971 
T.l. 


Spring,  1972 
T4 


i  N 

Experimental  LEA  #1  32 

Experimental  LEA  #2  31 

Experimental  LEA  #3  24 

Experimental  LEA  #4  29 

Control  LEA  #1  30 

Control  LEA  #2  36 

Total  |lo2 


8701 


SD 
H1.Q77 

.231 


It 

5.333p..007 
4. 3441.541 
4.5661.430 
4.4441.252 


N 

32 
23 
27 


20 


33 


.66 


X 


1^ 


5.00C 


3.962 


4.24; 


SD 


0.95 


1.55 


1.36  ) 


JTwo-Wav  Analysis 

LYar.. 

1         T3    6  T4 

Experiment  LEA#1 
W/Experiment  LEA#2 

1  ^ 

Signif.,, 

1  1.8fi^ 

Row 

1  S.fi91 

Experiment  LEA#3 
W/Experiment  LEA#4 

Col. 

1  1.870 

NS 

Row 

14.999 

.001 

Experiment  LEA#1 
tf /Control  I.FA#1 

Col. 

0.365 

NS 

Row 

7.452 

.01 

Experiment  LEA#2 
HtZControl  I.FA#1 

j 

0.065 

NS 

Row  1 

0.562 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

1.415 

NS 

Row 

13.392 

^  .001 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

1.302 

NS 

Row 

0.548 

NS  1 
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QUESTIONNAIRE  DATA 


Item  8 


I  think  that  the  oblectives  developed  during  AMA 

Clearly  statPd  wi      r-^^^^^  tn  rP^nH-.' 


Scale 
Values 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

■2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
 Tnral  


Two -Way  Analysis  of 
Variance 


Experimental  LEA#1  W/ 
Experimental  I.PA#? 


Experimental  LEA#3  W/ 
Experimental  LEA#4 


Fall,  1971 
T3 


N 


31 


2A 


7M 


116 


X 


5.709 


4.791 


SD 

n.fizni 


1.442 


i.R?n, 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Spring,  1972 
T4 


N 


32 


21. 


2£. 


111. 


X 


5.593 


4.692 


5  T, 


Col. 


Row 


Row 


0.896 


3.467 


0.082 


0.811 


Signif. 


NS 


JiS. 


NS 


NS 


SD 
J-J125. 


It316 
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QUESTIONHAIRE  DATA 


Item 


^  .  :  My  manager  makes  It  clear  that  he  is  committed  to  the 
success  oh  our  proiecEs:  ~   ^°  '^"^ 


SCALE 
VALUES 


ERIC 


ISxperiment  LEA#1  W/ 
Control  LEA  »1 
experiment  LEA#2  W/ 
Control  LEA  #1 

Experiment  LEA#3  W/ 
Control  LEA  #2 
txperiment  LEA#4  W/ 
Control  LEA  #2 
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qUESTIONKAIRE  DATA 


I  tea  2 


My  manager  has  expressed  the  belief  that  the  AMA's  training 
proaram  has  been  hf»lpful. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


hxperiment  LEA  #1 

Lxperiment  LEA  #2 

hxperiment  LEA  #3 

Hxoeriment  LEA  #4 

Total 


Fall  '70  T 


N 

28 
3B 

X 

5.451 

5.361 

124 

4.454 

4..'jlB 

1 

1  ."iRfi 


l.Sfl? 
2.180 
2.343 


Spring  '71 


N 

_23 
-26 
JJ 
-23 

91 


X 

f).240 
!i.4n 


SD 

1.984 
1.654 

1 .169 


Fall  '71  T, 


-29 
116 


6.'^n 


-S.87'^ 

5.172 


0  fi7 


Spring  '72  T4 


7 — 
.■3.4 -il 

■To- 
il^ 

1.34 

.■a.  075 

1.37 
1.62 

112 


Two- Way  Anal.ofVar. 

Tl  fi  t: 

2 

T2  §  T3 

T3  S  ' 

C4 

Tl  S  T4 

Kxperiitent  LEA#1  W/ 
Hxperiment  LEA#2 

F 

Signif. 

F 

Signif, 

F 

Signif. 

F 

Signif. 

Col. 

0.768 

NS 

8.091 

.01 

2. 146 

NS 

0.694 

NS 

Row 

0.262 

NS 

0.401 

NS 

14.364 

.001 

9.046 

.01 

Experiment  LEA#3  W/ 
Experiment  LEA#4 

Col. 

4.25fi 

.ns 

0.1B8. 

NS 

0.432 

NS 

4.617 

.05 

Row 

o.on? 

NS 

1.129 

NS 

2.  830 

NS 

0.253 

NS 
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QUESTIONNAIRE  DATA 


My  manager  understands  planning  thpnrv  is  ;,hiP 
to  t)ut  It  into  practice.   


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 
T3„ 


7.0 

6.8 

6.6 

6.4 

6.2 

tz  1 

•  ^' 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring,  1972 
T4 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


N 

32 

2A 
21 


X 


5.833 

5,222 


SD 

n.7Q 


1^: 


ILiif 


Q.38C 


N 

-23 

.12 
-21 

166 


X 

5.9.T 


5.3931.11 


SD 


Jwp-Wav  Analvsi5;  oi 

F  Var. 

T3    8  T4 

Experiment  LEA#1 
tf /Experiment  LEA#2 

P 

0.907 

NS 

Row 

2.80q 

NS 

Experiment  LEA#3 
W/Experiment  LEA#4 

Col. 

.05 

Row 

3,13? 

NS 

Experiment  LEA#1 
Hf /Control  !.RA#1 

Col. 

3.2?3 

Row 

3.33^ 

NS 

Experiment  LEA#2 
lit/Control  LFA#1 

1.029  . 

N? 

Row 

0.028 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

1.09  3 

NS 

Row 

3.059 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

1.738 

NS 

Row 

0.008 

NS 

Item 
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QUESTIONNAIRE  DATA 


to  do  my  work  effectively. 


ERIC 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Two-Wav  Analysis  oi 

par*. 

T3    g  T4 

Experiment  LEA#1 
W/Experiment  LEA#2 

P 

Signif. 

fiol, 

.  6.062. 

.05 

Experiment  LEA#3 
W/Experiment  LEA#4 

Col,  , 

0.177 

NS 

Row 

6,^57 

Experiment  LEA#1 
tf /Control  LEA#1 

Col. 

0.008 

NS 

Jlow 

2.877 

NS 

Experiment  LEA#2 
W/CDntrol  LEA#1 

Col. 

9.736 

NS 

Row 

0.708 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

0.004 

NS 

Row 

3.071 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

0.005 

NS 

Row 

1.014 

NS 

Fall,  1971 


X 

6.093 


5.322 


N 

32. 

22,14. g65 

3^ 

182 


M38 


SD 
0.856 


1.165 


Spring,  1972 


N 

22, 
22. 
21. 
2SL 
22. 


?j.437 


X 


4.fi.'il 


').7'^n 


5-?i? 


SD 

SIM 


1^6 


iL^7 


1*253 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2. 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


8 
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QUESTIONNAIRE  DATA 
feel  good  ahnut  mv  manager's  ahiiii-v  p]^n, 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 
— li 


Spring,  1972 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
<.8 
'♦.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 

r.4 

1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total  


N 

32. 

31 

2T 


0 

J2 


X 

6.437 


6.000 


5.875 


66 


5.805 


SD 
(LlII.5 


1.341 


0.899 


1.066 


N 

32 
23 

2Z 

20 


1.450 


33 


166 


X 


5.906 
5.347 


■^-444 


6.000 


SD 
SLM3 


1JJ7 
1.3-5 


J-^^8 


SLJJS 


0.92 


Two-Wav  Analysis  oj 

f  Yar. 

T3    6  T4 

Experiment  LEA#1 
W/Experiment  LEA#2 

■  Signifi 

Col. 

0.425 

,  NS 

^ow 

4.949 

.05 

Experiment  LEA#3 
W/Experiment  LEA#4 

Col. 

2.333. 

NS 

Row 

0.010 

N§ 

Experiment  LEA#1 
l)f/rontrol  LEA#1 

Col. 

0.fi3fl 

Row 

NS 

Experiment  LEA#2 

m/f.nntTn}  LEA#1 

Col. 

0.242 

NS 

Row 

2.766 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

0.542 

NS 

Row 

1.665 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

0.030 

NS 

Row 

1.721 

NS 

Item 
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QUESTIONNAIRE  DATA 

Mv  manappr  nrnyides  me  with  adequate  support  to 
perform  mv  job.   


ERIC 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3,2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Fall,  1971 


N 

22. 

31 
2± 
29 

0.82 


X 


5.709 
5.916 


5.793 
5.966 
5.861 


SD 
1l21 


1.39; 


0.97^ 


1.20( 


1.35! 


1.45 


Two-Wav  Analysis  o\ 

T3    8  T4 

Experiment  LEA#1 
W/Experiment  LEA#2 

P 

Signifi 

Col. 

0.023^ 

NS 

Row 

15.428 

.001 

Experiment  LEA#3 
W/Experiment  LEA#4 

2.193 

NS 

Row 

2.071 

NS 

Experiment  LEA#1 
tf/Control  I.F.A#1 

Col. 

0.000 

NS 

Row 

6.471 

.05 

Experiment  LEA#2 
IZControl  LEA#1 

Col. 

C.210 

NS 

Row 

0.956 

NS 

Experimeiit  LEA#3 
W/Control  LEA#2 

Col. 

0.195 

^5 

Row 

o.oir 

NS  ■ 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

i.gsr 

Row 

1.920 

NS 

Spring,  1972 


N 

2X 

32 
23 
27 


20 


33 


166 


X 

iai9| 


5.  843 


5.7 


5.787 


SD 
0x621 


1.08( 


821.2431 
T^5lT57! 


6.0500.99t 


1.13! 


3 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

S.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 
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QUESTIONNAIRE  DATA 


Item  1 


As  I  see  it,  planning  is  an  integral  part  of  running  the 
state's  schools.   •  


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.& 

5.6 

5.4 

5.2 

5.0 

4.  8 

4.6 

4.4 

4.2 

4.0 

3.6 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall  '70 

bxperiment  LEA  #1 
hxperiment  LEA  #2 
Lxpe rimer  LHA  #3 
l:xj)eriment  :a  #4 
Control  LCA  #1 
Control  LEA  #2 

Total  

Two-Way  Anal.ofVarl 


lixpcriment  LEA#1  WyCol 

Lxperiment  LEA#2  Row 

I-xperiment  LEA#3  W/jcol 

Lxperiment  LEA#4 


N 

31 
33 
27 


127 


X 

6,571 
6,384 
6.363 
6,481 


Tl  §  T2 


Row 


12.649 


1.086 


0.043 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Experiment  LEA#2  Wy 
Control  LEA  #1 


Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  W/ 
Control  LEA  #2 
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QUESTIONNAIRE  DATA 


As  T  ciPp  persons  in  thii 
f^ffnrt  inta  planning. 


^zation  put  a  lot  of 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total   


Fall,  1971 
T3, 


N 

_24 

182 


X 


5.32? 

6.000 


5-4QQ 
5,?77 


SD 


0.78C 


1.36C 


1^ 


Spring,  1972 
-14, 


N 

-22 

23 
27 


.13 


166 


X  SD 


5.521  1.12 


4.'37C  1.39 


'^-05C1.27 


U24 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8. 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Two-Wav  Analysis  of  Var 


Experiment  LEA#1 
W/Experiment  LEA#2 

Experiment  LEA#3 
W/Experiment  LEA#4 


Experiment  LEA#1 
Hf/Control  LHA#1 


Experiment  LEA#2 
li[/rnntro1  LEA#1 


Experiment  LEA#3 
W/Control  LEA#2 


Experiment  LEA#4 
W/Control  LEA#2 


T3 


6  T4 


£oL 


Q.967 


3.571 


Col. 


8.412 


Row 


15.454 


Col. 


2.565 


4.348 


CoL 


Row 


0.84^ 


0.071 


Col. 


0.917 


Row 


4.257 


Col. 


3-171 


Row 


4.087 


NS 


,001 


NS 


NS 


NS 


NS 


,05 
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QUESTIONNAIRE  DATA 

My  capability  to  plan  effectively  will  positively 
affect  my  future  career  in  this  organization. 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 


SCALE 
VALUES 


4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 


Fall,  1971 


5 

X 

6 

.161 

5 

.958 

6 

.i75 

5 

,733 

6 

.138 

Spring,  1972 


X 


6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2. 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  LEA  #1 

Expert 'nental  LEA  #2 

Expei..:..-ntal  LEA  #3 

Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total  


N 

32 
31 
24 

in 

-30 

,36 
182 


SD 


0.82( 


1.39f 


0.64{ 


N 

32 
23 
27 


iai( 


1.24; 


-22 

33 


166 


5.87C 


ILZiC  -6 


5.782  1.2 


5.88i 


SD 
J^(8 


1.2!i0 

19 


Two-Wav  Analysis  oi 

I  Var. 

T3    6  T4 

Experiment  LEA#1 
W/Exoeriment  LEA#2 

P 

Signif . 

Col, 

1.549 

.    MS  . 

Jlow 

MS 

Experiment  LEA#3 
W/Experiment  LEA#4 

fpl, , 

1  .(inn 

Row 

0.8517 

Experiment  LEA#1 
If /Control  LEA#1 

Col. 

o.m? 

Row 

Nfi 

Experiment  LEA#2 

W/rontrol  LEA#1 

Col. 

NS 

Row 

0.404 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

0.566 

NS 

Row 

0.643 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

2.127 

N? 

Row 

0.021 

NS 

Item 
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QUESTIONNAIRE  DATA 


The  activities  relating  to  planning  are  having  an 
effect  on  the  policy  or  this  organization^ 


ERIC 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 


SCALE 
VALUES 


3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total  


Two-Wav  Analysis  oi 

f  Var. 

T3    6  T4 

Experiment  LEA#1 
W/Experiment  LEA#2 

F 

Sigrnif . 

Col. 

0.841 

NS 

How 

11.  4  •5.-? 

.001 

Experiment  LEA#3 
W/Experiment  LEA#4 

£.2,1,  . 

7.037, 

Row 

0rP4? 

NS 

Experiment  LEA*1 
W/Control  LFA#1 

Col. 

4.536. 

.05 

Experiment  LEA#2 
klZControl  LEA#1 

Col. 

o.iqn 

NS 

Row 

1.557 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

1.529 

NS 

Row 

3.459 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

3.041 

NS 

Row 

2.43?5 

.  NS 

Fall,  1971 

__JE3 


N 

32 


31 
24 
2? 

H 
_36 

182 


X 

6.218 
5.387 
5.625 


5.7l^ 


5.733 
4.972 


SD 
0.75d 


1.45; 


0.92 


TToTi 


1.66^ 


Spring,  1972 


N 

31 


21 
23 


20 


31 


166 


X 

5.87 


C 


5.375 


5.13( 


5.60C 


4.83£ 


SD 
SU9 


1.1! 


1.3- 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

S.O 

4.8 

4.6 

4.4 

4.2. 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 
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APPENDIX  B 


SCHEMATIC  PRESENTATION  OF  DATA  -  INTERVENING  VARIABLES, 

CHAPTER  FIVE 


550 


QUESTIONHAIRE  DATA 


^^^'^  '  -  Based  on  information  7  have  r^r^^w^^   ^oss    I  know  if  t 

am  measuring  up  in  iwy^  ■        ^  doss^  t  Know  if  i 


Experiment  SED#1 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


ERIC 


551 


QUESTIONNAIRE  DATA 


My  manager  encourages  and  supports  innovation. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
S.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1970 

T1 


Spring,  1971 

T9 


Fall,  1971 

T1 


•  E'L 


Spring,  1972 

TA 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


N 

39. 
60. 
66. 

L65 


X 

IG.051 


5.900 
5.712 


SD 
0.999 


N 

73 


X 

6.137 


SD 

1.045 


N 

68 


X 

5.97dl 


SD 
.184 


N 

39 


X 

5.666 


SD 
1.382 


1.189 


1.274 


52. 
61 


5.135 
4.836 


1.534 
1.675 


45 


67 
6T" 


5.02C1.581 
5.22:1.515 


42. 


61 


186 


4.150 
4.983 
5.166 


1.477 


1.454 


1.342 


Two-Wav  Anal  of  Var. 

Tl    g  T2 

T2    g  T3 

T3  fi 

T4 

Tl    g  T4 

Experiment  SED#1  W/ 
Control  SED  tl 

F 

P 

^ignif. 

.     F  .. 

Signif . 

Sign 4  f  

Colt 

22.841 

.001 

0.541 

NS 

2.143 

NS 

8.430 

'.01 

Row 

5.304 

.05 

38.695 

.001 

14.799 

.001 

7.658 

.01 

Experiment  SED#2  W/ 
Control  SED  #1 

1.727 

NS. 

0.424 

NS 

6.677 

.01 

15.758 

.001 

Row 

19.668 

.001 

0.053 

NS 

6.281 

.05 

2.965 

NS 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

2.204 

NS 

Row 

7.574 

.01 

Col. 

6.645 

.01 

Row 

11.238 

.001 
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QUESTIONNAIRE  DATA 


Item  L    Higher  management's  reactions  to  the  problems  that  reach  them 

are  tair.  


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


Fall,  1970 
-JlL 


N 

65 


X 

5.436 
4.950 
5.492 


SD 
1-535 


1.512 
1.002 


Spring,  1971 
—X2. 


21 


49 


58 


X 

5.677 


4.673 


5.138 


SD 

1.18Q 


1.420 
1.420 


Fall,  1971 
— li. 


N 


45 


67 


61 


X  SD 
5.3821.414 


5.0661.420 


5.089a.287 


5.287a. 362 


Spring,  1972 


N 


61 


65 


38 


SD 


4.42:.  1.4(i3 


4.8611  1.4  .9 
T7?5(l  1.3(i9 


Two-Wav  Anal  of  Var. 

Tl    S  T2 

T2    6  T3 

T3  fi 

T4 

Tl    6  T4 

Experiment  SED#1  W/ 
Control  SED  #1 

F 

 E  

p 

p 

-Signif .  . 

Coll 

0.113 

NS 

1.416 

NS 

3.707 

NS 

2.005 

NS 

5.819 

.05 

5.704 

.05 

2.013 

NS 

Experiment  SED#2  W/ 
Control  SED  #1 

8;072 

.01 

0.633 

NS 

4.882 

.05 

9.847 

.01 

Row 

3.170 

NS 

2.604 

NS 

1.441 

NS 

7.284 

.01 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

0.712 

NS 

Row 

1.710 

Col. 

6.054 

.05 

Row 

4.509 

.05 
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QUESTIONNAIRE  DATA 


IteqLj_   Mv  manager  knows  anrf  nndprs^anris  thc^  pirrOrjl  ^ms  I  face. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
S.8 
5.6 
5.4 
5.2 
.  5.0 
4.8 
4.6 
4.4 
4.2 
4, 
3, 
3, 
3, 
3, 


.0 
.8 
.6 
.4 

.2 


3.0 
2.8 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Fall,  1970 
-10. 


N 

39 
59 
65 


163 


SD 

0.992 


X 

5.  58!> 
5.42:!  1.392 
5.70t  1.377 


Spring,  1971 
—12. 


N 

73 


52 


X 

5.410 


14.365 
5.343 


SD 
1.279 


1.559 


Fall,  1971 
— Xi 


N 


45 


61 


241. 


X 

4.888 


SD 


e-z. 


Spring,  1972 


N 
205 


X 

fi.  3.131 


SD 
J-J. 


4.qfifi   1  41 


4.742  J-J.f4 


Two -Way  Anal  of  Var. 


Tl    S  T2 


T2    6  T3 


T3 


q  T4 


Tl    6  T4 


Experiment  SED#1  W/ 
Control  SED  »1 


£21 


2.313 


0.020 


Signlf 


NS 


0.122 


NS 


0.104 


Si 


NS 


0.925 


7.299 


NS 


1.347 


NS 


0.453 


'01 


NS 


Experiment  SE0#2  W/ 
Control  SED  #1  


.001 


1.504 


NS 


4.304 


05 


21.144 


,001 


Row 


10.967 


.001 


12.671 


,001 


6.152 


.05 


4.762 


,05 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


Col. 


0.308 


NS 


Row 


6.496 


:o5 


Col, 


7.934 


Row 


0.750 


NS 
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QUESTIONNAIRE  DATA 


My  manager  recognizes  when  a  problem  is  developing  and  does 
something  constructive  about  it.  r.  j  


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
.  5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.C 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Two-Wav  Anal  of  Var. 


Fall,  1970 
-ILL 


N 

39 
60 
ST 

164 


X 

5.53d 


5.33E 


SD 

1.166 


4.966  1.495 


1.361 


Spring,  1971 


N 

73 


52 
64 


132. 


Tl    6  T2 


X 

5.31 

T.  98  m 


4.578 


SD 

1.091 


1.577 


1.660 


T2    6  T3 


Fall,  1971 
—X3. 


N 

68 
45 

24T 


X 

4.97d 


4.78d 


SD 

1^ 


3..539-m 


J^7M6 

201 


T3  fi 


T4 


Spring,  1?72 


N 

215  39 


1^  OP  1*225 


X 

5.17? 


4.5qn    -  t^QR 


4.424 


SD 


Tl    S  T4 


Experiment  SED#1  W/ 
Control  SED  #1 


7.588 
6.884 


Signif. 


01 


,01 


0.041 
5.835 


0.050 


8.327 


,05 


3.393 


NS 


4.247 


Signif. 


01 


.05 


Experiment  SED#2  W/ 
Control  SED  #1 


Col  ■  .9.600 


,01 


3.127 


NS 


5.573 


,05 


tow 


6.037 


25  .  046 


.001 


,05 


5.235 


,05 


7.498 


.01 


8.704 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


.01 


Col. 


0.137 


Row 


5.146 


NS 


US- 


Col. 


Row 


6.035 
5.144 


.05 


,05 
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QUESTIONNAIRE. DATA 
Item  ^      My  manager  shows  confidence  and  trust  in  me. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 


2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Fall,  1970 
-XL 


N 

_39 
60 

164 


X 

6.282 


6.050 
67500 


SD 
0.759 


1.213 


1.018 


Spring,  1971 
—12- 


N 

73 
52 
62 


187 


X 

6.178 


5.326 


5.758 


SD 
1.109 


1.424 


1.237 


Fall,  1971 


N 


45 


67 


61 


241 


X 

^.058p. 


SD 


5.8000.990 
5.8501.018 


5. 55711.466 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

CI 

5.6 

5.4 

El. 

5.2 

5.0 

4.8 

4.6 

4  4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring,  1972 
^1:4- 


N 


205J 


X 

223. 


405.225 


615.622 


SD 

I. lay 


1.57() 


1.25:i 


Two-Wav  Anal  of  Var. 


Tl    S  T2 


T2    6  T3 


T3  S 


T4 


Tl 


6  T4 
— 7 — 


Experiment  SED#1  W/ 
Control  SED  #1 


3.656 


3.092 


■Signif . 


NS 


0.010 


NS 


5.614 


Si  oni  f . 


NS 


1.126 


NS 


8.022 


NS 


3.639 


NS 


1.934 


.HI. 


JiS_ 


Experiment  SED#2  W/ 
Control  SED  #1 


Col. 


13.516 


001 


3.176 


NS 


Row 


3.363 


NS 


17.244 


2.304 


NS 


1.768 


NS 


2.634 


NS 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


Col, 


Row 


8.547 
O.iiB 


NS 


Col, 


Row 


2.644 
0.02§ 


NS 
1^ 
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CONTENT  ANALYSIS  DATA 

 1  :  Involvement  in  Decision-Making  in 


Exp.fl 


H= 4.813  : Sig. ■  .05 
p"  0.000  ; Sig . ■  NS 


Control  #1 


the  st-a^e  Depart nipnt; , 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

mh,JMO  Spring.  1972 


H"2.037  ;Sig."  NS 


pO.OOO  ; Sig . -  NS 


H-J-ieo  :Sig.-  NS 


p"  0.000  ;Sig.«  NS 


Exp.#l 


-o 


H«9.334  ; Sig . «  .01 
p  »0.000  ; Sig . «  NS 


Control  #1 


FRIC 


Control  #1 


Fall,  1970 


Exp.#2  ^ 

hJlU Sig."  NS 
p=O.OCO  ; Sig. ■  NS 


H"1.213Sig.«  NS 


p a  0.000  Si g .  =  NS 


H«  1.160  Sig. a  NS 


p «  0.000  Si g . ♦  NS 


Fall,  1970 


Exp.  #1 


H«2.03  '  ;Sig.«  NS 


H-1-333  ; Sig . »  NS 


p*0.000  ; Sig. ■  NS 


P »  0.000  ; Sig . n  NS 


H»1.213  ;Sig.,  NS 


Exp. #2 


p*0.000   ;Sig.»  ^ 


Spring,  1972 
 Q  Exp. #2 


Ha  1.131  ;  Sig.  g  NS 
P  ~  0.  000  ;Sig.a  NS 


Control  #1 


Spring,  1Q79 

 0 


Exp.#l 


Ha7.259  ; Sig . «  .01 
p  aO.OOO  ;Sig.a  NS 


Exp. #2 
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It—        :    Involvement  in  Decision-Making  in  the  State  Department 


Faii,  1970 

N 

Kruskal-Wallis  One-Nay 
Analysis  of  Variance 

Binominal  Test  of 
Proi^ortions 

Experiment #1 
ContXDl  #1 

12 

H-  4.813 

P=  0.000 

12 

Signif.*  .05 

Signif.=  NS 

Experiment#2 
Control  #1 

12 

H«  1.470 

pa  0.000 

12 

Signif.«  NS 

Signif.=  NS 

Experiment #1 
Experiment#2 

12 

H»  1.333 

p^O.OOO 

12 

Signif.«  NS 

Signif.=  NS 

Spring,  1972 

Experiment* 1 
Control  #1 

8 

H«  9.334 

P=  0.000 

11 

Signif.*  .01 

Signif.=  NS 

Experiment #2 
Control  #1 

9 

H= 1.131 

P=  0.000 

11 

Signif.*  NS 

Signif.=  NS 

Experiment #1 
Experiment#2 

8 

H=  7.259 

P*  0.000 

9 

Signif.=  .01 

Signif.=  NS 

Fail,   1970  ti 
Spring,  1972 

>\ 

Experiment  #1 
Experiment  #1 

12 

Hx  2 .037 

p=  0.000 

8 

Signif.=  NS 

Signif.«  NS 

Experiment  #2 
Experiment  #2 

12 

H-1.213 

p=  0.000 

9 

Signif.»  NS 

Signify  NS 

Control#l 

Control#l 

t  — 

12 

H=  1.160 

P  =0.000 

11 

Signif-*  NS 

Signif.=  NS 
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ooNTarr  analysis  data 

Item_^___:  Quality  of  Decision-Makincr  in  the  State  Department. 


SCHEMATIC  PRESENTATION  OP  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.«l 


Exp.#l 


H"  0.789  ; Sig . ■  _NS 


H=1.141  : Sig.  ■  NS 
p=0.000  ; Sig . «  NS 


Control  #1 


p«  0.000  ; Sig . -  NS 


H"  5.761  : Sig . .02 


H»  4.854  ; Sig . g  .05 
P  *  0.000  ;Si8.»  NS 


p"  0.000  ;Sig. 


NS 


Control  #1 


Fall,  1970 


Spring,  1972 


Bxp.#2 

H=6.901  ; Sig. »  .01 
P=0.000  ; Sig . ■  NS 


a«lj^23  Sig.«  NS 


Control  #1 


p  =  r:_000_Sig.  =  NS 


O 


Exp,*2 


Ha 0.763  ; Sig .  =  NS 
p  =  0.000  ;Sig.=  NS 


H«  5/761  Sig.  =  .02 


p « 0.000  Sig . ♦  NS 


Control  #1 


ERIC 


Fall,  1970  Spring.  1972 

E;^.  #1  H«0.789  ;Sig.g  NS 


H»1.763  ; Sig . ■  NS 


D*0.000  ; Sig . «  NS 


\ 
i 

;-Exp.#2 


p=  0.000  .Sig. a  NS 


H>L.823  ;Sig..  NS 


paC.OOO  ; Sig . «  NS 


O 


Exp.#l 


H=7.253  ;Sig . "  .01 
p  aO.OCO  ;Sig.=»  NS 


0 


Exp. #2 
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It—  2       Quality  of  Decision-Making  in  State  Department. 


Fall,  1970 

N 

Kruskal-Wallis  One-Wa/ 
Anal/sis  of  Variance 

Binominal  Test  of 
Proportions 

Experiment#l 
Control  #1 

12 

H" 1.141 

P  =  c.ooo 

12 

Signif."  NS 

Signil,=  NS 

Experiment#2 
Control  #1 

12 

H- 6.901 

p  =  0.000 

12 

Signif.«  .01 

Signif,-'  NS 

Experiment#l 
Experiment#2 

12 

H« 1.763 

Pi^O.OOO 

12 

Signif."  NS 

Signif, =  NS 

Spring,  1972 

Experiment#l 
Control  #1 

9 

H-4.854 

P=  0.000 

11 

«!ignif.«  .05 

Signif, =  NS 

Control  #1 

9 

H=0  763 

»     u . uuu 

1 1 

J-  J- 

cxpc  rinien  u  i 
Experiment#2 

9 

Ha  7   2  53 

p*  0  or>r> 

9 

Fall,   1970  to 
Spring,  1972 



Experiment  #1 
Experiment  #1 

12 

H*0,789 

p= 0.000 

9 

Signif. «  NS 

Signif. «  NS 

Experiiaent  #2 
Experiment  #2 

12 

H-i-823 

P=  0.000 

9 

Signif. a  NS 

Signifr  NS 

Control#l 
Control#l 

12 

H-5.761 

P  =  0.000 

 ].1_ 

Signif  •»  ,02 

Signif. =  NS 

ERIC 
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tf 


OOWTBfT  ANALYSIS  DATA 

^ tg»  3    ;  Influence  of  Planning  on  Decision-Making  Prnr.^f,„ 

SQiEMATIC  PRESPfTATION  OP  ANALYSTS 


Bxp.fl 


H'11.430j  Sig. ■  .001 
p"  l.OOO; Sig . ■  Ns 


Control  #1 


Fall,  1971 


H" 0.048  ;Sig.«  NS 
P"  1.000  ; Sig . ■  NS 


H" 0.000  ; Sig . ■  NS 


p ■ 0.030  ;Sig . ..05 


Spring,  1972 


Exp.#l 


H"  l0.922Sig.-  .001 
P* _JLl9£?Sig..  IC 


Control  *1 


ERIC 


Fall,  1971 


Exp. #2  ^ 

H0.447  ; Sig . -_NS_ 
P=1.000  ; Sig .  ■  NS 


H- 1.120  Sig..  NS 


Control  #1 


p  =»  1.000  Sig .  -  NS 


Spring.  1972 
 Q  Exp. 92 


H-0.000  Sig. "  NS 


p '0.030  Sig. ♦  .05 


H- 1,433  ; Sig. «  NS 
P  "I'OOO  ;Sig.-_^ 


Control  #1 


Fall.  1971 


Exp.  #1 


H-0.048  tSig. ■ 


H«11.430;  Sig .  ■  .001 


^*  1.000  ; Sig . -  NS 


:Sig.< 


Spring.  1972 

 0 


H«1.120  ; Sig .  •  NS 


B]qp.«2 


Exp.n 


H-2.940  ;Sig.»  NS 
p  =1.000  ; Sig . a  NS 


•  P«'--000  ;Sig.- 


Exp. #2 
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It—     3  :   Influence  of  Planning  on  Decision-Making  Process 


Fall,  1971 

N 

Kruskal-Wallis  One- Way 

Binominal  Test  of 
r  xopoxx  xons 

Experiment #1 
Control  #1 

9 

H-11.430 

P  =1.000 

11 

Signify-  .,,01 

Signif.=  NS 

Experiment #2 
Control  #1 

10 

H«  0.447 

p  =1.000 

11 

Signif.*  NS 

Signif.=  NS 

Experiment#l 
Experin6nt#2 

9 

H:il.430 

1.000 

11 

Signif.«  .001 

Signif.=  NS 

Spring,  1972 

- 

Experiment#l 
Control  #1 

9 

HJ.0.422 

P =1.000 

11 

Signif.«  .001 

Signif.=  NS 

Experiment#2 
Control  #1 

10 

H=1.433 

P  =1.000 

11 

.  Signif.*  NS 

Signif.=  NS 

Experinient#l 
Experiment#2 

9 

H=»  2.940 

P*1.000 

10 

Signif.«  NS 

Signif.=  NS 

Fall,   1971  to 
Spring,  1972 

Experinent  #1 
Experiment  #1 

9 

H« 0.048 

p=  1.000 

9 

Signif.=  NS 

Signif."  NS 

Experiment  #2 
Experiment  #2 

10  ■ 

H-1.120 

p= 1.000 

10 

Signif.»  NS 

Signify?  NS 

Control#l 

Control #1 

f.  ,   .  ,  — 

8 

Hs-O.OOO 

P  =0.030 

10 

Signif.«  NS 

Signif.=  .05 
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QUESTIONNAIRE  DATA 


ItenL 


JE2i£.  -people  I  work  with  participate  appronriatelv  in  spf-t-^ng 
the     pals  of  our  worV.  


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 


3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SHD  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Two-way  Anal  of  Var. 


Experiment  SED#1  W/ 
Control  SED  #1 


Fall,  1970 


N 

39 
60 

IL65 


X 

769 


5.167 

5Tm 


SD 
0.93i 


1.460 
1.26S 


Spring,  1971 
_X2- 


N 

72 


51 


60 


183 


CqL 


Tl    6  T2 


7.002 


7.582 


Signif 


.01 


X 

5.375 


4.804 

43n 


SD 
1.305 


342 
568 


T2    6  T3 


0.700 


01  .1.138 


■Signif  ■ 


,001 


11.139 


Fall,  1971 


N 

68 


45 


67 


241 


X 

5.485 


4.822 


4.  970 


SD 
1.094 


1.48;  39 
1.3i:  59 


T3    5  T4 


2  .708 


■Si  an  if 


,001 


Spring,  1972 
-14- 


N 

40 


204 


SD 

5.250  l.llt3 


4 


384  1.515 

2 


66]|  1.37 
T 


Tl 


T4 


12.471. 


7.653 


001 


01 


IdO"- 


Experiment  SED#2  W/ 
Control  SED  #1  


0.844 


NS 


0.215 


NS 


3.539 


NS 


13.495 


,001 


Row 


5.154 


05 


0.158 


NS 


NS 


_NS_ 


Experiment  SED#1 -W/ 
Control  SED  #2 
Experiment  SEn#2  W/ 
Control  SED  #2 


Col. 


1.870 


Row  |i'  .270 


NS 


.UUi 


Col. 


2.716 


Row 


1.04.1 


NS 
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QUESTIONNAIRE  DATA 


ItenL 


lam  appropriately  involved  in  decisions  affecting  my  work. 


SCALE 
VALUES 


.  7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 

.  5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SFO  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


Fall,  1970 


N 

39 
59 
66 


X 

5.949 


5.661 
5.879 


SD 
0.887 


1.385 
1.117 


Spring,  1971 
_X2- 


N 

73 


X 

5.699 


52. 
if 


14.808 


5.082 


SD 
1.298 


1.456 


1.452 


Fall,  1971 


N 

68 


X 

15.441 


67 


61 


45  5.177 


5.298 


5.147 


SD 
I.42d 


1.336  40 
.61 


1.3U 


1.52E 


Spring,  1972 
-14- 


N 

40 


207 


X 

5.5751 


4.950  I 


7.0 


2 
0 
8 
6 
1.4 
1.2 
1.0 


0.9(34 


4.950  1.55 


4€|6 

3 


4.9B4  1.420 


Two-Wiv  Anal  of  Var. 

Tl    8  T2 

T2    8  T3 

T3  fi 

T4 

Tl    6  T4 

Experiment  SED#1  W/ 
Control  SFD  #1 

F 

Signif. 

F 

Signify. 

P 

Signif . 

p 

.c:ignif 

4.355 

'.05 

0.012 

NS 

0.344 

"NS 

14.408 

.001 

10.125 

.01 

5.287 

.05 

4.431 

.05 

3.858 

.05 

Experiment  SED#2  W/ 
Control  SED  #1 

COI. 

1.S59 

NS 

2.526 

NS 

2.091 

NS 

19.479 

.001 

Row 

22.038 

.001 

1.108 

NS 

0.933 

NS 

0,384 

NS 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

0.006 

NS 

Row 

5.662 

.05 

Col. 

0.923 

NS 

Row 

0.000 

NS  J 
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QUESTIONNAIRE  DATA 


,1  can  influence  the  goals-,  methods,  and  ach j u i ^- ^ oc  of 
my  organization.  ~"  yx^xes  or 


Scale 
Values 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 


Two-Wav  Analysis  of  v^v 


Experiment  SED#1 


Experiment  SED#2 
W/Control  SPn#] 


Experiment  SED#1 
W/Contrm  qppffy 


Experiment  SED#2 
W/Control  SPno^ 


£aL 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

£1 

5.0 

4.8 

4.6 

ct 

4.4 

tl. 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall,  1971 
T3 


N 
68 

45 

67 

61 
1241 


X 

5.220 


4.577 


4.731 


4.770 


SD 

1.43J 


1.46S 


1.342 


1^1 


0.356 


9-060 


.224 


1.013 


0.520 


8.276 


1.807 
0.972 


JA. 


Spring^^l972 


N 

40. 

M 

61 
I  07 


X 

5JLL5 


SD  ^ 


1.5^ 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
6 
2.4 
2 
0 
1.8 
6 
4 
2 
0 


NS 


NS 


NS 


.01 


NS 
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QOWrBNT  ANALYSIS  DATA 


^ tern  1  ; Promote  Cooperative  Team  Work, 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H=  1.477  ;Sig. 
p=  0.736  ;Sig. 

Exp. #2 


NS 


NS 


Q 


H«2.318  ; Sig.  NS 


p  =>0.02     :Sig;  .05 


H-9.800  ; Sig . «  .01 


p  ^0.141  ; Sig.  =__NS_ 


Exp.#l 


H= l0.50aSig.=  .01 
P=   0.18aSig.=  NS 


-6 


Exp. #2 


Fall,  1970 

N 

Kruskal-Walli§  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Pmpn-rtinnc 

Experiment  #1 
Experiment  #2 

8 

7 

.4=1.477 
Sig."  NS 

P -0.736 
Sig.  NS 

Spring,  1971 

Experiment  #1 
Experiment  #2 

8 

7 

H»l 0.500 
Sig-=  .Oi 

p  =0.180 
Sig.=  NS 

Fail,  1970  t 
Sprinq,  1972 

o 

Experiment  #1 
Experiment  #1 

8 
8 

H=2.318 
Sig.-  NS 

•P=  0.02  . 

sig-»  .n-i 

Experiment  #2 
xperiiaent  #2 

7 

H"9.800 
Sig.-  .01 

p =0.141 
Sig.=  NS 
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otman  analysis  data 

Item  2     ; Amount  of  Cooperative  Teamwork  Present, 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.fl 


H=0.059  : Sic •  ■  NS 
p=0>189   : Sig . «  NS 


Control  fl 


H»4.321  ;Sig.»  .05 
p«0.302  ; Sig . ■  NS 


H- 0^337  : Sig . -  NS 


p "0.012  ;Sig . ■  .05 


Exp.#l 


O 


H=  0.000  : Sig .  =  NS 
p  »0.026  :Sig«=  .05 


Control  #1 


Fall,  1970 


Bxp.#2  Q 

H=0.77l  ;  Sig. »  NS 
p=O.020   : Sig . «  .05 


Control  #1 


H«0.466  Sig. «  NS 


p g0.l04  Sig.  =  NS 


HxO.337  Sig.=  NS 


p =0.012  Sig . »  .05 


Spring,  1972 


Exp. #2 


H=2.062  ; Sig.  =  NS 
p  =0.000  ;Sig.= -QQl 
002 

Control  *1 


Fall,   1970  Spring,  1972 

Exp.  #1  H«4.32l  :Sig.=  .05 

 o 


H»l,929   : Sig.  =  NS 


p»0.736   ;  Sig . «  NS 


Exp.*2 


0.302  ;Sig.»  NS 


H"0.466  ;Sig.«  NS 


p«0.104  ;Sig.=  NS 


Exp.#l 


H=3.490  : Sig.  =  NS 


p  =0.028  ;Sig.  =  .05 


0 
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It4 


Amount  of  Cooperative  Teamwork  Present. 


ERIC 


Fall,  1970 

N 

Kruskal-Wallis  One-Way 
Analysis  of  Varicince 

Binominal  Test  of 
Proportions 

Experiment #1 
Control  «i 

7 

H«0.059 

P  =0.189 

5 

Sisnif.*  NS 

Signif.=  NS 

cxperlMntf  2 
Control  #1 

Q 

o 

'7*7  1 

p  »  0. 020 

5 

Sisnif.*  NS 

Signif.s  .05 

Experiinenttl 
Experi]iient#2 

7 

Urn  1  n9n 

8 

SlEnif.-  NS 

Signif.=  NS 

Spring,  1972 

Experimenttl 
Control  #1 

4 

H«  0.000 

P=0.026 

2 

Signif.-  NS 

Signif.=  .05 

£xperiment#2 
Control  #1 

8 

H=  2.062 

P=0. 000002 

2 

Sijjnif.*  NS 

Signif.=  .001 

Experiment#l 
Experii&ent#2 

4 

H»  3.490 
Signif."  NS 

P<0.028 

8 

Signif.=  .05 

Fall  1970  to 
Spring,  1972 

Experiaent  #1 
Experinent  #1 

7 

H-4.321 
Slanif.-  .05 

p=0.302 

4 

Signify*  NS 

Experiaent  #2 
ExperiMnt  #2 

8 

H«  0.466 

P=0.104 

8 

Signlf.-  NS 

Signify  NS 

Control*! 
Controlfl 

5 

H-0.337 

P  "0.012 

2 

Sl£nif.-  NS 

Signif.s  .05 
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QUESTIONNAIRE  DATA 


Item  3       My  group  works  hard  to  achieve  its  ana  I  s 


SCALE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Total  


Two-way  Anal  of  Var. 


Experiment  SED#1  W/ 
Control  SED  #1 


Experiment  SED#2  W/ 
Control  SED  #1 


Fall,  1970 
-11. 


N 

39 
60 
66 

161 


X 

6.128 


5.634 
6  .091 


SD 
0.656 


1.149 


0.956 


Spring,  1971 
— -X2~ 


N 

73 


52 


60 


{L85 


Row 


Experime.it  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


Tl    6  T?. 


.81; 


5.298 


qol. 3 .016 


13.841 


NS 


T05^ 


NS 


X 

5.932 


5.538 


5.600 


SD 
0.855 


1.019 


1.437 


T2    S  T3 


0.436 


TTSTT 


1.101 


Q.214 


Signif 


NS 


TIT 


NS 


JiS. 


Col. 


Row 


Col. 


Row 


Fall,  1971 


N 


T 


68  5.955 


45  5.711 


61 


67  5.776 


5.721 


SD 
1.12^ 


1.057 


1.112 


1.12-/ 


T3 


T4 


3.01 


■Signif 


Trrr 


NS 


IS" 


12.175 


6.113|  .05" 


l-7Rai  MR 


1.50f 


NS 


0.81! 


NS 


4.16: 


.05 


2.67< 


NS 


Spring,  1972 


N 

40 


39 


60 


66 
205 


X 

5.675 


5.17S 


5.533 


5.651 


SD 
l.Oll 


l.O'^e 
l.ie5 


!.95 


Tl    6  T4 


0.374 


11.581 


7.4179 


.001 


NS 


,001 


7.0 
6.3 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
6 
2.4 
2.2 
0 
1.8 
1.6 
1.4 
1.2 
1  0 


ERIC 
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QUESTIONNAIRE  DATA 

Item  4        My  work  group  understands  what  we  are  trying  to  achieve. 


SCALE 
VALUES 


4.8 
4.6 

4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 


Fall,  1970 


Spring,  1971 
— X2- 


Fall,  1971 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

B' 

5.6 

5.4 

5.2 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Spring,  1972 
—14- 


Experimeni.al  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


N 

66 
165 


X 

5.667 


5.317 
5.727 


SD 
0.898 


1.172 


N 

J2. 


49 


1.075 


61 


183 


X 


5.143 


SD 

Q.928 
1.208 
5.164  1.306 


N 


45 


241 


X 

5.8ne 

5.400 

5.43.2 
5.573 


SD 


N 

AD 


1729* 


66 


206 


SD 


5.19(,  1.1(^6 


5.36;  1.2:?3 


Two-Wav  Anal  of  Var. 

Tl    6  T2         '       T2    &  T3 

T3  g 

14 

Tl    S  T4 

Experiment  SE0#1  W/ 
Control  SED  #1 

p 

Signif . 

Si£ni f  ' 

P 

F 

Si'gnif... 

1.395 

NS 

4 . 384 

.05 

2.805 

NS 

3.399 

N5 

Row 

5.483 

.05 

6.777 

.01 

5.704 

.05 

1.337 

NS 

Experiment  SED#2  W/ 
Control  SED  fl 

1.^16 

NS 

3.164 

NS 

8.288 

.01 

11.563 

.001 

Rcw 

5.594 

.05 

0.140 

NS 

2.610 

NS 

6.915 

.01 

Experiment  SED#l'  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  JEr^  #2 

Col. 

1.682 

NS 

llow 

'2.n^ 

Ny 

Col. 

5.872 

.05 

Row 

4.900 

.05 

Item  ^ 
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QUESTIONNAIRE  DATA 

I  feel  my  group  works  well  togeth^i;- 


SCALE 
VALUES 


5.2 
5.0 


3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1970 
T1 


Spring,  1971 
—12- 


Fall,  1971 
— Xi 


Spring,  1972 
-34- 


Hxperimental  SED  #1 
experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
rptaJ  


N 

39 
59 

|163 


5.794 
5.271 
5.738 


SD 
0.863 
1.215 
1.162 


N 

73 


51 


64 


188 


X 

5.  57  5 


4.862 


5. 187 


SD 
0.998 


1.311 
1.166 


N 

& 


SD 


N 


5.344aT5Tr 


15?. 589 


-£1^295 
66B.151 


2^ 


SD 
1.3l!i 


Two-way  Anal  of  Var. 


Tl    6  T2 


T2    6  T3 


T3  6 


T4 


Tl  T4 


Experiment  SED#1  W/ 
Control  SED  #1 


7.339 


2.439 


Signi.f 


01 


6.984 


NS 


6.181 


Signif ■ 


.01 


3,355 


NS 


3.603 


05 


3.197 


NS 


1.056 


Signif ■ 


NS 


JiS- 


Experiment  SED#2  W/ 
Control  SED  #1  ■ 


Col. 


9,.3Q.^ 


,01 


Row 


6.342 


4.773 


1 .  91Q 


-US. 


05 


JIS. 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED<'2  W/ 
Control  SED  #2 


4.369 


Col , 


2.062 


NS 


Row 


8.098 


,01 


Col. 


1.855 


Row 


1.015 


MS 

'ns 
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QUESTIONNAIRE  DATA 

It^   6      I  really  feel  my  immediate  work  group  is  getting  things  done 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
?.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 

UJ  


Twn-Wav  Anal  of  Var. 


Experiment  SED#1  W/ 
Control  SED  #1 


Fall,  1970 
_X1. 


N 

60 
65 

164 


X 

,897 


5.800 
5.830 


SD 
0.882 


1.021 


1.024 


Spring,  1971 


N 

73 


52 


64 


189 


Ti6T2 


Col. 


0-955 


2.234 


S-ig 


nif . 


NS 


NS 


X 

5-904 


5.384 


5.546 


SD 
0-915 


1-239 


1-344 


T2    6  T3 


0-107 


4.703 


Sift 


nif. 


NS 


.05 


Fall,  1971 
— Xi, 


N 

68 


45 


67 


61 


241 


X 

5.882 


5.G22 

5.656 
5.590 


SD 
1.01$ 


1.19 


1.13:;  60 


1.07(t>  65 
204 


T3  6 


3-384 


2-110 


T4 


Sign 


NS 


NS 


Spring,  1972 
— T4- 


N 

40E.600 
395.153 


5.400 
5.415 


SD 
0.92$ 

1.38(» 

1.23' 

l.io: 


Tl    6  T4 


5.810 


0.779 


11.742 


■if. 


.05 


NS 


Experiment  SED#2  W/ 
Control  SED  #1  


5.397 


,05 


1.105 


NS 


4-462 


Row 


0.411 


NS 


0-354 


NS 


0.659 


NS 


0.776 


NS 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED»2  W/ 
Control  SED  #;J 


Col. 


Row 


2-698 
2-935 


NS 


Col. 


Row 


3.791 
0.452 


NS 
fJS" 


Item. 
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QUESTIONNAIRE  DATA 


•|^tgiii|'t£lS"ggr§L'vls;"  "^^  ^"^^^  ^^"^^P'  ^^^'^  way^^  for 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 


8 
6 
4 
2 
0 
8 
2.6 
2.4 
2.2 
2.0 
1.8 
1,6 
1.4 
1.2 
1.0 


Fall,  ,1970 
-11. 


Spring,  1971 


Fall,  1971 


Spring,  1972 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 
Tata]  


N 

39 
60 
65 


164 


X 

5.717 
5.300 
5.676 


SO 

0.971 


N 

73 


1.356 
1.047 


51 


64 


188 


X 

5.534 


4.921 


5.265 


SD 

0.92  9 


N 

68 


1.453 


45 


1.287 


67 

■6T 


241 


5.09 


SD 
1. 


3  79 


750  1.4D9 


4.95  )  1.3  58 


09  )"T72  35 


Two-way  Anal  of  Var. 


Tl    6  T2 


T2    6  T3 


T3.  & 


T4 


Tl    6  T4 


Experiment  SED#J  W/ 
Control  SED  #1 


Col.  4.372 


Sianif  ■ 


.05. 


0.000 


NS 


3.328 


■Signif 


NS 


3.326 


NS 


2.774 


NS 


2.141 


NS 


0.728 


■Signif, 


NS 


NS 


Experiment  SED#2  W/ 
Control- SED  #1  


^.^23 


Row 


4.687 


0.047 


NS 


2.104 


NS  .3.386 


.001 


,05 


2.694 


NS 


1.429 


NS 


2.732 


NS 


Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 


Col, 


Row 


0.875 


NS 


TI5" 


Col. 


Row 


0.426 
1.241 


NS 


NS 
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QUESTIONHAIRE  DATA 


Item    1  ;  Based  on  information  I  have  received  from  my  boss. 

I  know  if  I  am  measuring  up  in  my  -fr.v>  


SCALE 


VALUES 


ERIC 


Experiment  LEA#1 
Control  LEA  #1 
Experiment  LEA#2  W/ 
Control  LEA  #1 
Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  W/ 
Control 
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QUESTIONNAIRE  DATA 

Item     2  :   My  manager  encourages  and  supports  innovation 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.1 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall- '70  T, 


Spring  '71  T 


Fall  '71  T, 


Spring  '72  T4 


7.0 
6.8 

6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
i.S 
1.6 
1.4 
1.2 
1.0 


l.xpuriment  LEA  #1 
! xperiment  LEA  #2 
l.xpcriment  LEA  #3 
i. xperiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


N 
28 
39 
33 
27 


L27 


X 

6.250 
5.923 
5.969 
6.074 


SD 
0.751 
1.285 
0.983 
1.206 


N 

1' 
2i 
\ 

23 


X 

5.240 
5.953 


SD 
1.690 
1.137 
1.169 
1.780 


N 
_32 
_31 
JA 

-2- 

_36 
182 


T 

,40d 


.06£ 

.33; 


SD 
0 


6IJ4  3]| 


6 

6.09d  0.90r7  33  6.0612  1 . 
5.95q  0.90f7  23  5.73P  1 
6 

6 


1. 


146  3; 


16- 


T 

6. 


12  9 


£05 

6.06E  l.C 14 
5.739  1.355 
5.481  1.E53 
i^U^C50 
6. 151  0.795 


SD 
0.: 


IVo-Way  Anal.ofVar, 


Tl  5  T2 


gjgnif. 


T2  e  T3 


T35T4 


Tl  S 


T4 


Experiment  LEA#1  W/|Col 
experiment  LBA#2  |row 


Hxperimont  LEA^S 
l:xperiment  LEA#4 


W^Col 


4.261 


0.918 
2.700 


.05 


9.888 


NS 


1.258 


Signif 


.01 


1.061 


NS 


1-546 


Signif, 


NS 


Q.Q02 


NS 


1.2Q7 


JiS_ 


NS 


2.510 


NS 


Row 


0.001 


NS 


0-000 


NS 


2.824 


NS 


Q,.Q86 


JE- 


n.P9.7.. 


JJS_ 


Experiment  LEAt'l  W/ 
Control  LEA  #1 


Gol 


0.179 


NS 


Row 


3.947 


Experiment  LEA#2 
Control  LEA  #1 


Col 


0.038 


NS 


Row 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Or796 


Experiment  LEA#4  W/ 
Control  LEA  #2 


Col 
TTow 


1.000 


NS 


TIoT 


3.858 


Row 


3.351 


JJS. 


4.945 


.05 
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QUESTIONNAIRE  DATA 


Higher  managements  reaction  to  the  problems  which  reach 
them  are  fair.  ,  „  


SCALE 
VALUES 


LEA 
LEA 


#3 
#4 


I  XMcriroent  LEA  #1 
'AiHriment  LEA  ^2 
t  Apcriment 
i.xic  rlment 
t  Ml  vol  LEA  n 
(ontrol  lc^  #2 

rolal  

T'.  o- Way^Anal .  ofVar . 


ixperiment  LEA#1  W/jcoT 
• xpcrtment  LEA#2 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

ei 

5.6 

5.4 

5.2 

5.0 

ei 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

~'xfK*rinient  LEA^3 
rxi)cnTient  LEA#4 


".xpcrir.-ient  LEA^l  W/ 

rontrol  LEA  ^1   

i  vperimiTnt  LEM2  W/ 
Control  LEA  #1 
^ixperimenv  LEA^3  W/ 
>  ontrol  LEA  ^2 
llVperiiXent  LEA#4  W) 
Control  LEA  fr2 
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QUESTIONNAIRE  DATA 


Item    4  ;  My  manager  knows  and  understands  the  problems  I  face. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
'5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Lxperiment  LEA  #1 
hxperiment  LEA  #2 
l.xpcriment  LEA  #3 
Experiment  LEA  #4 
(.ontrol  LEA  #1 
Control  LEA  #2 

Total  

Two-tVay  Anal.ofyarT 


lixperiment  LEA#1  W/ Col. 

Hxperiment  LEA#2  Row 

l-xperiment  LEAff3  W/|col 

Experiment  LEA#4 


F.xperiment  LEA#1  W/ 
control  LEA  #1 


lixperiment  LEA#2  W/ 
Control  LEA  #1 


lixperiment  LEA#3  W/ 
Control  LEA  #2 
Hxperiment  LEA#4  W/ 
Control  LEA  #2 


Fall  '70  T, 


N 
28 
39 
33 


27 


.27 


X 

6.035 
5.512 
5.575 
5.407 


SD 
0.999 
1.315 
1.275 
1.308 


Row 


Tl  S  T2 
F 


Spring  '71  T, 


N 

25 
26 
17 
23 


91 


11.195 


0.000 


0.680 


0.004 


Signif. 


.001 


NS 


NS 


NS 


X 

4.640 
5.192 
5.176 
?.3C)4 


T2  6  T3 


SD 
1.845 
1.265 
1.286 
L.^.87 


20.69C 


1.227 


1.175 


0.15C 


Signif 


,001 


NS 


NS 


NS 


Col 


Row 


Col 


Row 


Col 
ftow 


Col 


Row 


Fall  '71  T, 


T 

32 
31 
24 
22 
M 
36 
82 


X 

5.000 
5.967 
5.583 

5.555 


T3  5  T4 


SD 
0.9821 


0.795 


LJUJ 


1.382 


3.011 


0.036 


3^4_ 


PtQOQ 


3.846 


2.035 


1.567 


3,. 089., 


2.796 


0.064 


3.937 


0.059 


Signif, 


NS  0.142 


NS 


JiS_ 


NS 


NS 


NS 


NS 


Ji§_ 


J15. 


NS 


Spring  '72  T4 


N 

32 


21 


J2JL 
4-211 
11 


161 


5.6 


560 


5.7500.87S 


■§D 

.970 


1  .ifie 

LjBjQI 


1.341 


Tl  S  T4 


.394 


■  ■^90 


Signif. 


NS 


NS 


-NS., 


7.0 
6.8 

6.6 
6.4 
6.2 
6.0 
5.8 
5:6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


ERIC 
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QUESTIONNAIRE  DATA 


Itea    5  ;  My  manager  recognizes  when  a  problem  is  developing  and 
does  something  constructive  about  it.   


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
S.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3r2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Kxperiment  LEA  #1 
Experiment  LEA  #2 
Experiment  LEA  #3 
Experiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total  

Two-Way  Anal.ofVarT 


Fall  '70  T, 


N 

28 
39 


33 


27 


127 


X 

5.964 
5.461 
5,$?6 
5.4Q7 


SD 
0.744 
1.210 
1.103 
1.474 


Tl  §  T2 


Spring  '71  T, 


N 

25 
26 

13 
23 


91 


Experiment  LEA#1  W/jCol. 
Experiment  LEA#2 


Experiment  LEA#5  W/|Col. 
Experiment  LEA#4 


Experiment  LEA#1  W/ 
Control  LEA  #1 
Experiment  LEA#2  W/ 
Control  LEA  #1 


Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  W/ 
Control  LEA  #2 


Row 


1.538 


0.104 


4.086 


0.344 


Signif. 


.001 


NS 


^5 
N3* 


X 

4.760 
5.115 
5.560 
4.956 


T2  §T3 


SD 
1.640 
1.117 
1.215 
1.637 


F      I SignTT 


13.016  .001 


0.000 


3.464 


0.031 


NS 


NS 


Gol 


Row 


Col 


Row 


Col 
Row 


Row 


ERIC 


Fall  '71  T3 


T 

32 
31 
24 


29 
30 
36 
182 


X 

.937 
,580 


■  541 

■  482 
.566 

■  388 


SD 

0.87 


1.111}  32 


1.06 
1.27 


1.43 


T35T4 


F 

1.415 


0.706 


6.601 


0.037 


2.105 


1.491 


0.270 


0.251 


3.480 


0.114 


3  .360 


0.296 


Signif. 


NS 


,05 


NS 


NS 


NS 


NS 


NS 


JJS- 


jas- 


NS 


Spring  '72  T4 


31 


23 


L27 
>  20 


)  33 


167 


5.51^ 


 1 


5.15 


SD 

1.: 


!1 
.4 

4.82d  1.4)7 


4.771  1.6)1 
5.35(t  1.348 

>5 


1.3 


Tl  §  T4 


0.867 


1.498 


7.633 


0, 


Signif 


NS 


.01 


NS 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
3.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 


3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 
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QUESTIQNHAIRB  DATA 


Itatt  6    :        n>anager  shows  confidence  and  trust  in  me. 


SCALE 
VALUES 


hxperiment  LEA  #1 
lixperiment  LEA  #2 
hxperiment  LEA  #3 
Experiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Two-Way  Aral.ofVaT 


Uxperiment  LEA#1  Wy 
I:x])eriment  LEA#2 


Experiment  LEAff5  W/jCol 
Hxperiment  LHA#4 
lixperiment  LEA#1  wy 
Control  LEA  #1 
Experiment  LEA#2  wy 
Control  LEA  #1 
Experiment  LEA#3  Wy 
Control  LEA  #2 
Experiment  LEA#4  wy 
Control  LEA  #2 
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QUESTIONNAIRB  DATA 


Item   ;  The  people  I  work  with  participate  appropriately  in 

settingL  the  goals  of  our  work.   


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
S.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall  '70  T, 


Spring  '71  T 


Fall  '71  T, 


 T" 

5.875 
5.483 

5.J15D 
5.310 

5.S33 
5.083 


Spring  '72 


TT 

31 
32 


5.67' 


bxperiment  LEA  #1 
I;xperiment  LEA  #2 
Lxperiment  LEA  #3 
l-.xperiment  LEA  #4 
Control  LGA  #1 
Control  LEA  #2 
Total 


N 

28 
29 

23 

27 


127 


Two-way  Anal.ofVar 


X 

.500 


5.076 

5.090 
5.148 


Tl  5  T2 


SD 
0.962 
1.222 

1.2R3 
1.511 


N 

25 
26 


U 

23 


91 


X 

.040 


4.423 
4.695 


T2  S  T3 


SD 
1.567 
1.474 

0.882 
1.917 


32 

_31 

24 
29 

JD 
26 
tL82 


SD 

0.97 


1.38 


0..-7->t 
1.13 


_23 
26 


1.19! 
1.33t 


I  20 
33 


167 


T3  5  T4 


5.18- 


4.76« 


'^■fi'ia 
4.96 


SD 
0.6$2 


1.2! 


1.6110 

_1-D1I9 
1.1115 


Tl  S  T4 


Signi^. 


lixperiment  LEA#1  W/ 
Experiment  LEA#2 


Col 


Row 


Experiment  LEAff3 
Experiment  LEA#4 


Row 


5.185 


4.520 


0.370 


0.494 


Signif . 


05  13.885 


.05 


3.925 


NS 


4.957 


NS 


2.297 


Signif 


.001 


1.573 


5.007 


5.094 


NS 


2.425 


Signif, 


NS 


0.583 


5-865 


n..3!76 


NS 


NS 


jia. 


JiS. 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Gol 


0.045 


NS 


Row 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Col 


0.951 


NS 


Row 


0-140 


NS 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Col 
Row 


2.550 


Experiment  LEA#4  W/ 
Control  LEA  #2 


Row 


3.247 


Ji5_ 


NS 


1.830 


NS 


0.003 


NS 


580 

QUESnONNAIRB  DATA 


Item  2 


I  am  appropriately  involved  in  decisions  affecting  mv  work. 


ERIC 


SCALE 
VALUES 


7  n 

O  ^  9 

O*  O 

A  A 

O  •  Z 

ft  n 

«>  •  o 

et 

1%  ^ 
9  •  o 

^« 

^  •  *♦ 

^  •  V 

d  ft 
**  •  o 

A  f% 
•t  •  o 

A  A 

A  0 

A  n 

3.8 

3.6 

S.4 

3.2 

S.O 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall  '70  T, 


Expeximent  LEA  #1 
l-Ixperiment  LEA  #2 
hxperiment  LEA  #3 
i'xperiment  LEA  #4 
Control  LEA  #1 
control  LEA  #2 

Total 
Two- Way  Anal.ofVaF!! 


N 

28 
39 
33 

2J. 
|L27 


Hxperiment  LEA#1 
hxperiment  IJA#2 
Experiment  LEA#3 
Hxperiment  LEA#4 


W/C01.L4.277 


Hxperiment  LEA#1  W/ 
Control  LEA  »1 
Hxperiment  LEA#2  W/ 
Control  LEA  #1 


Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  W/ 
Control  LEA  #2 


X 

5.964 
5.769 
5.303 
5.666 


SD 
.035 
1.157 
1.530 
1.3QQ 


Row 


Row 


Tl§  T2 
"~F  


Spring  '71  T 


N 

25 
26 
17 

91 


2.200 


0.82  9 


0.058 


Signif . 


.001 


NS 


NS 


NS 


X 

5.200 
4.653 
5.294 


T2§  T3 


SD 
1.707 
1 . 440 
1.358 
1^,222. 


13.390 


3.838 


4.468 


Q.123 


NS 


JJS. 


Col 


Row 


Col 


Row 


Col 
Aow 


Row 


Fall  '71  T 


TT 

32 
31 
24 
25 

82 


T3  5  T4 


3 

0.87C 


1.44: 


o.9ie 


1  -OQf 


0.566 


4-114 


n.04'i 


0.156 


0.001 


0.023 


1.551 


n-06Q 


0.231 


Signif, 


jas. 


NS 


JiS. 


NS 


Spring  '72  T4 


TT 

31 


32 


2J 

Jn. 

167 


"T" 

5.9031 


5.562  1.1 


5.60(1 


5.391  1.11 


■sr 

0.943 
4 

7 

JL2i3 
JLJ.^2 


Tl  S  T4 

[Signif. 


-US- 


-NS- 


0 
.8 
.6 

4 

2 

.0 
.8 
.6 
.4 
.2 
.0 
8 
.6 
4 
.2 
.0 
.8 
6 
4 
2 
.0 
8 
.6 
.4 
.2 
.0 
.8 
.6 
4 
.2 
.0 


Item 
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QUESTIONNAIRE  DATA 

I  can  influence  the  goals,  methods,  and  activities 
of  mv  organization. 


ERIC 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 

6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total   


Two-Wav  Analysis  of  VarJ 

T3    6  T4 

Experiment  LEA#1 
W/Experiment  LEA#2 

  F  . 

£ol, 

0.040 

NS 

4.766  ' 

.05 

Experiment  LEA#3 
W/Experiment  LEA#4 

1.417 

^   

Row 

0.528 

NS 

Experiment  LEA#1 
t|l/Control  LEA#1 

Col. 

NS 

Ra» 

1.397 

NS 

Experiment  LEA#2 
tf/r.ontrol  LEA#1 

Col. 

0.020 

NS 

Row 

0.515 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

0.043 

NS 

Row 

0.617 

NS 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 
Row 

0.378 

NS 

o.OOQ 

NS 

Fall,  1971 


N 

32 


X 

5.750 


21 

29 

M 

36 

n.82 


5.322 

5._S17 

5.138 


SD 
0.87 


1-221 


1.121 


0.98"? 


1:312 


N 

31 


Spring,  1972 
T4 

X 

5.774i 


-22 

27 


2Sl 

33 


166 


S-iL5  ±^<c  3 


SD 

0.8(J4 


5.000  1.6€ 4 


J-^4 
l.J^3  iJ.€7 
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QUESnOltMXIB  DATA 


Item  1 


My  group  works  hard  to  achieve  its  goal. 


SCALE 
VALUES 


hxperiment  LEA  #1 
Experiment  LEA  #2 
Uxperiment  LEA  #3 
Experiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Tvvo-Way  Anal.ofVar 


Experiment  LEA#1 
Experiment  LBA#2 


Experiment  LEA#3 
Experiment  LEA#4 


Experiment  LEA#1  W/ 
Control  L£A'#1 
Experi.nent  LEA#2  W/ 
Control  LEA  #1 
Hxperiment  LEA#3  W/ 
Control.  L.iA  #2 
Experiment  LEA#4  W/ 
Control  LEA  #2 
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QUESTIONNAIRE  DATA 


Item     2  ;   My  work  group  understands  what  we  are  trying  to  achieve. 


LE 
VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 
5.6 
5.4 
5.2 

CM  ^ 

5.0 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4" 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

I  xperiment  LEA  #1 

Experiment  LEA  #2 

I  xperiment  LEA  #3 

I. xperiment  LLA  #4 
(.ontrol  LEA  «1 
Control  LEA  #2 

lotal   


Fall  '70  T, 


N 

28 
39 
33 
27 

L27 


Iwo-Way  /vnal.ofVar. 


i:xpcriment  LEA#1  W/lCol 
lixDcriment  LEA#2 


ixpcriment 
i-xperiment  LEAfS 
Experiment  LEAff4 


X 

5.642 


5.435 
5.303 


5,444 


SD 
.869 


.231 


.9P3 


Spring  '71  T. 


N 

25 
26 
12 
21 


91 


X 

4.760 
4.653 


Tl  5  12 


Row 


12.386 


0.438 


Signif. 


,001 


NS 


T-  6  T3 


SD 
507 


1.412 


0.951 
1.789 


Fall  '71  T, 


N 

32 
31 
24 
29 
2D 
36 
.82 


X 

5  ■  968 
5.322 

5J12A 
6.J123 
5.^72 


SD 
0.8971 


1.491 


n.7q: 


1  ,nqF 
n.7iF 

1.38 


13.92]  -001 


2.235  ^ 


Signif 


T3  §  T4 

[Signif. 


0.014 


7.1.39 


NS 


0.453 


Spring  '72  T4 


31 


21 


J3. 


-26 

3i 


167 


5.9031 


A22 


.652 


3110 


Td 

0- 


-LJiJO 


Tl  S 


T4 

Signif. 


NS 


jaa. 


5. 


0 

8 

6 
4 
2 
0 
8 
6 
4 
2 
0 
8 
6 
4 
2 
0 
8 
6 
.4 
2 
0 
8 
6 
4 
2 
.0 
.8 
.6 
.4 
.2 
.0 


0 
8 
4 

LjQ^9 
xl^£7 


W^CoT 


0.450 


NS 


Row 


0.704 


NS 


NS 


n-683 


JIS_ 


1  .114 


JJS- 


1-296  NS 


0.209 


NS 


0-000 


NS 


Experiment  LEA#1  Yi/ 
Control  LEA  #1 


Gol 


2.958 


NS 


Row 


0.818 


NS 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Col 


0.669 


NS 


Row 


3.341 


NS 


Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  w/ 
''Mitrol  LEA  #2 


Col 


Row 


0.052- 


_NS_ 


Col 


0.392 


0.004 


Row 


0.040 


NS 
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QUESTIONNAIRE  DATA 


Item     3  :   I  feel  my  group  works  well  together. 


ERIC 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


faxperiment  LEA  #1 
lixperiment  LEA  #2 
Experiment  LEA  #3 
Experiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Tivo-Way  Anal.ofVaF 


Hxperiment  LEA#1  W/ICoL 
Experiment  LBA<>2  |Row 


Experiment  LEA#3 
Experiment  LEA#4 


Experiment  LEA#1  W/ 
Control  LEA  #1 


Experiment  LEA#2  W// 
Control  LEA  #1 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Experiment  LEA#4  W/ 
Control  LEA  #2 


Fall  '70  T, 


N 

28 

39 

23 
27 


127 


X 

5.642 
5.205 
5.515 
5.444 


SD 
0.869 

1.004 
1,093 
1-423 


Ti  5  T? 


Spring  '71  T 


T 

25 
26 

JJ 

23 


91 


Row 


6.612 


2.647 


0.027 


0.767 


Signif. 


.05 


NS 


NS 


NS 


~3r" 

5.439 
4.730 

5.217 


T2  S  T3 


SD 
1.581 

1.185 
0.996 
1.881 


19.427 


1,149 


3.058 


1.494 


Signif 


.001 


NS 


NS 


NS 


Gol 


Row 


Col 


Row 


Col 
rtow 


Row 


Fall  '71  T, 


T 

32 
31 

23 
IP 

0.82 


— T" 

6.000 
5.774 
6.JJ:ii 
5.793 
6 . 066 
5.638 


T3  S  T4 


1.04 


I.IK 


Q.75I 


0.90 


1.33^r  33 


2.615 


2.167 


0.3.53. 


2.975 


0.171 


2,?^6 


3.392 


0.035 


4.473 


0-188 


0.022 


Signii^i 


NS 


NS 


NS 


NS 


NS 


NS 


NP 


■  05 


_NS- 


NS 


Spring  '72  T4 


31 
'32 


i_23 
2S> 


2Q 


5.74 


5.31; 
5..57e 


7.0 
6.8 
6.6 
6.4 

6.2 
6.0 
5.E 
5.(> 
5.4 
5.2 
S.O 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 


1.4 
1.2 
1.0 


1.315 
1.4(.8 
n.949 
_L-2'0 

_LJ)ii9 
Q.8!i3 


T15T4 


0.242 


4-272 


2  .  309 


1.570 


Signif. 


NS 


_NS_ 
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QUESTIONNAIRE  DATA 


4  :   I  really  feel  my  immediate  work  group  is  getting  things  done. 


SCALE 
VALUES 


Lxperiment  LEA  #1 
hxperiment  LEA  #2 
Lxperiment  LEA  #3 
lixperiment  LEA  #4 
Coittrol  LEA  #1 
Contiol  LEA  #2 

Total  

Tu'b'Way  Anal.ofVar 


1.0^6 

1^:9 


l-'xperiment  LEA#1 
hxperiment  LEA#2 


Hxperiment  LEA#3 
[Experiment  LEA#4 


Experiment  LEA#1  W/ 
Control  LEA  ^1 
lixperiment  LEA#2  W/ 
Control  LEA  #1 
Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4  Wy 
Control  LEA  #2 


586 

QUBSnONNAtkS  DATA 


Item    5  ;  vnien  differences  arise  in  my  work  group^  we  have  good 
ways  for  aettilina  them  ourselves.  


SCALE 


VALUES 


hxperiment  LEA  #1 
i.xperiment  LEA  #2 
Experiment  LEA  #3 
Experiment  LEA  #4 
Control  LEA  ^^1 
Control  LEA  #2 

Total 
Two- Way  Anal.ofVar. 


Bxperiment  LEA#1  W/|coI 
Experiment  lflA*2 


Experiment  LEAffT 
Experiment  LEA#4 


ERLC 


Experiment  LEAtl  W/ 
Control  LEA  #1 
Experiment  LEA#2  W/^ 
Control  LEA  #1  . 
Experiment  LEA#3  W/ 
Control  LEA  #2 
Experiment  LEA#4 
Control  LEA  #2 


587 


APPENDIX  C 


SCHEMATIC  PRESENTATION  OF  DATA  -  END-RESULT  VARIABLES, 

CHAPTER  SIX 


588 


QorfrBwr  analysis  data 

I  tern  1  ;  Produce  and  Implement  a  Long  Range  Plan 


SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,  1970  Spring,  1972 


Exp.#l 


H«0.534  ;Sig.  NS 


H=  0.633  -.Sig.g  NS 


P=  0.0Q2  :Sig.*  ni 


Exp. #2 


p  ^.005  :Sig;  .01 


6- 


H"10.140eSig . «  .01 


Exp.fl 


H«  2 . 868  :  Sig.  =__N 
P=  0.430  ;Sig.=  Nl 


p  a  0.10  tSig.a  NS 


Exp. #2 


Fall,  1970 

N 

Kruskal-Wallii  One-Way 
Analysis  of  Variance 

Binomial  Test  of 

Experiment  #1 
Experiment  #2 

5 

H=0.633 

P  #0.002 

10 

Sig.«  NS 

Sig.  .01 

Spring,  197: 

Experiment  #1 
Experiment  #2 

7 

H-2 . 868 

p  =0.430 

9 

Sig.=  NS 

Sig.=  NS 

Fall  1970  tc 
Spring,  1972 

- 

Experiment  #1 
Experiment  #1 

5 

HaO.5340 

,pal0.1400 

7 

Sig.«  NS 

Sig.=  .01 

Experiment  #2 
}q>eriment  #2 

10 

H«0.005 

p=0.10 

9 

Sig.-  .01 

Sig.«  NS 

589 

QUESTIONNAIRE  DATA 


Iteio  2_     My  or(^aniif.ation ' s  overall  plan  is  operable. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1970 


Spring,  1971 


Fall,  1971 


Spring,  1972 
-14- 


Experimental  SED  #1 
Experimental  SED  #2 
Control  SED  #1 
Control  SED  #2 


N 

39 
164 


X 

.079 


4.2Q3 
5.2Q3 


SD 
1.132 


N 

73 


l.fi48 
1.21Q 


188 


X  , 
4.821 


4.0% 


4.5QQ 


SD 
1.456 


N 

68 


1.375 

1-447 


-15. 
£0. 


241 


5.411 


4-7.37 


,SD  ,^ 
1.31$ 

1.4P 
1.23 
1.66 


5o 


£6 


205 


5^17$ 


2-'8 


4.25e  J^(i3 


4.54!  1.3:i7 


Two-Wav  Anal  of  Var. 

Tl    5  T2 

T2    fi  T3 

T3  g 

T4 

Tl    6  T4 

Experiment  SED#1  W/ 
Control  SED  #1 

F 

Signif . 

p 

7  .207 

.01 

8.802 

.01 

2.175 

NS 

1.782 

Row 

0.300 

NS 

6 .  368 

.05 

9.328 

.01 

1.400 

NS 

Experiment  SED#2  W/ 
Control  SED  #1 

9.421 

.01 

4.795 

.05 

Row 

1.705 

NS 

1.207 

NS 

Experiment  SED#1  W/ 
Control  SED  #2 
Experiment  SED#2  W/ 
Control  SED  #2 

Col. 

1.290 

NS 

Row 

^.914 

,9Q1 

Col. 

3.327 

NS 

Row 

0.242 

NS 

^     •  Number  of  objectives  toward  which  Progress  has  been 


made. 


Fall,  1971 

N 

Analysis  of  Variance 

Binominal  Test  of 
Proportions 

Experiment*! 
% 

Control  #1 

9 

H-1.213 

P  =  1.000 
Signif. a  NS 

11 

Signlf.-  NS 

nxperxmenvw^ 

Control  #1 

xu 

H«0 . 019 



P  « 1.000 

Signif. =  NS 

11 

Signif.-  NS 

Experiment*! 
Experiment #2 

9 

H"  1  211 

Sicnif.s  \-rc* 

11 

Signif."  NS 

Spring,  1972 

Experiment#l 
Control  #1 

9 

H-3.753 

Sigr.if.s  NS 

11 

Signif.-  NS 

experiment 
Control  #1 

10 

H=0 . 044 

Signif. =  NS 

11 

Signif.*  NS 

Experiment* 1 
Experiment* 2 

9 

H»3 . 081 

Signif. =  NS 

10 

Signif.*  NS 

Fall  1971  to 
Spring,  1972 

Experiment  *1 
Experiment  *1 

9 

H-  0.779 

P=  i.OOO 
Signif."  NS 

9 

Signif.*  NS 

Experiment  *2 
Experiment  *2 

10 

H-  0.005 

p=  1.000 
•Sxgnify  NS 

10 

Signif."  NS 

Control*! 

Control*! 

I?  

11 

Signif."  NS 

P  "  l.QOQ, 
Signif. s  NS 

11 

ERIC 


T 
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OONTPIT  ANALYSIS  DATA 
I  tern    1   ;  Number  of  objectives  toward  which  Progress  has  been  made. 

SCHEMATIC  PRESENTATION  OF  ANALYSIS 

Fall,   1971  Spring,  1972 


Exp.il 


H=  1.213  :  Sig ..  ■  nS 
p=  1.000  : Sig .  «■  NS 


Control  #1 


H^.779  ;Sig."  NS 
P"l .  000   ; Sig . «  NS 


H»  0.052  ; Sig . »  NS 


P *  1.000  ; Sig . »  NS 


Exp.#l 


o 


H=3  /753  : Sig •  =  NS 


p  »1,000  : S ig ,  =  NS 


Control  #1 


Fall,  1971 


Exp. #2  Q 

H=0.019  : Sig . ^  NS 
p^l.OOO  : Sig . «_NS__ 


Control  #1 


H«  0.005  Sig . «  NS 


p g  1,000  Sig .  =:  NS 


H«  0,052  Sig.=:  NS 


p  =^1.000  Sig . »  NS 


Spring,  1972 

 0 


Exp.  #2 


H=  0.044  ;Sig.«  NS 
P  =  1>000  ;Sig.g  NS 


Control  #1 


ERIC 


Fall,  1971  Soring.  1972 

Exp.  #1  H» 0^779  : Sig.  =_NS 


H»1.213  ;Sig. »  NS 


0*1.000  ;Sig . »  NS 


P '  ^-000  •  Sig .  =  NS 


H-O5O05  ; Sig . «  NS 


Exp.«2 


p .1.000  ; Sig . g  NS 


Exp.#l 


H=3  .081  :Sig.=  NS. 


P  "l-OOO  ;Sig.=  NS 


6 


Exp. #2 
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OONTEMT  ANALYSIS  DATA 

Item  2    ;  Level  of  Progress  toward  those  Objectives. 


SfflEMMIC  PRESBKTATION  OP  ANALYSIS 

Fall,  1971  Spring,  1972 


Exp.«l 


H"  1.996  ; Sig . «  NS 


Exp.«l 


H=1.213  :  Sig.  -_NS_ 
p=1.000  ;  Sig . «  NS 


Control  #1 


p«  1.000  ; Sig .  B  NS 


o 


H«0>107  ;Si8.«  NS 
p «1>000  ; Sig. «  NS 


H»  4.688  : Sig. «  .05 
p  »  1.000  :Sig.^  NS 


Control  #1 


Fall,  1971 


Exp. #2 


H=0>447  ;  S ig .  m  NS 


Sig.' 


p=l>000    ;  Sig.  •  NS 


Control  #1 


Sig.= 


H=  0.107  Sig.  =  NS 


p si.  000  Sig . »  NS 


Spring,  1972 


Exp. #2 


HziO.lOO  :Sig.s 


NS 


p  si. 000  ;Sig.g  NS 


Control  #1 


ERIC 


Fall,  1971 


Exp.  #1 


H»1.213  ; Sig . «  NS 
p»1.000  ;Sig.«  NS 


Exp.»2 


H«1.996  ; Sig . «=  NS 


p a  1.000  ; Sig .  a  NS 


H-1.000  ; Sig . a  NS 


p»1.000  ;Sig. 


NS 


Spring,  1972 

 Q 


Exp.#l 


H=4.335  ; Sig.°  .05 
p  =  1.000  ;Sig.''  NS 


Exp. #2 


T 


It—  2    ;  Level  of  Progress  toward  tnose  Objectives . 


ERIC 


Fan,  1971 


ExperlMnti-1 
Control  #1 


Bxperijienti2 
Control  #1 


Experiaentil 
Experimenti2 


Spring,  1972 


E^qperinenttl 
Control  #1 


ExperiiDent#2 
Control  #1 


Experimenttl 
Experiment#2 


Pall  1971  to 
Spring,  1972 


Experiment  #1 
Experiment  #1 


Experiment  #2 
Experiment  #2 


Controltl 
Controltl 


N 


11 


10 


11 


11 


11 


10 


11 


10 


10 


10 


11 


11 


Kniskal-Wftllis  One-Way 
Analysis  of  Variance 


H«  1.213 
Slgnif.-  NS 


H>  0.447 


Slgnif."  NS 


H«  1.213 
Slgnif.-  NS 


H>  4.688 


Slgnif."  .05 


H«  0.100 


Signif.*  NS 


H«  4.335 


Slgnif."  .05 


H*  1 . 996 
Signif. -NS 


H«  1.000 


Signif."  NS 


H"  0.107 
Slgnif.*  NS 


Binominal  Test  of 
Proportions 


Pa  1.000 


Signif.: 


NS 


.  1.000 


Signif."  NS 


p» 1.000 
Signif."  NS 


P=  1.000 
Signif. =  NS 


P=1.000 


Signif. °  NS 


P*1.000 


Signif. «  NS 


1.000 


Signif."  NS 


1.000 


Signify  NS 


P  ° 1-000 
Signif..  NS 


QUESTIONNAIRE  DATA 


A3  I  see  it,  my  organization  has  made  progress  in 
af-.t-ainina  H-s  r>>.npr.4-i Tra»  


Scale 
Values 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 
T3 


Spring^^l972 


Experimental  SED#1 
Experimental  SED#2 
Control  SED#1 
Control  SED#2 


45 
2^ 


5.^411 


4.866 


SD  , 
1.34d 


1.34; 


207 


5.^2_5 
"5 


lain 


0.8$8 


5.114  J^:  9 


Two-Wav  Analysis  0 

P  Var. 

}     T4  . 

Experiment  SED#1 
W/Cnntrnl  SF.nfl  , , 

Signif.  , 

rnl 

0.504 

NS 

1.314 

NS 

Experiment  SED#2 

W/Control 

rnl , 

,0.852 

NS 

1.248 

NS 

Experiment  SED#1 
W/Control  SRn#2 

0.430 

NS 

4.271 

.05 

Experiment  SED#2 
W/Contr9l  §ED#2 

Col. 

0.726 

NS 

Row 

4.596 

,05 

Item  1 
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QUESTIOHHAIRE  DATA 


My  organization's  overall  plan  is  operable. 


ERIC 


SCALE 


VALUES 


7.0 

6.8 

6.6 

6.4 

6.2 

6.0 

5.8 

5.6 

5.4 

& 

5.2 

5.0 

et 

4.8 

4.6 

4.4 

4.2 

4.0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

2.4 

2.2 

2.0 

1.8 

1.6 

1.4 

1.2 

1.0 

Fall  '70  T 


bxperiment  LEA  #1 
hxperiment  LEA  #2 
l.xperiment  LEA  #3 
hxperiment  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 
Total 


Two-Way  Anal.ofVar. 


lixperiment  LEA#1  W/  Col 
lixperiment  LEA#2  Row 
Experiment  LEA#3  V/^oT. 


Experiment  LEA#4  [Row 
Experiment  LEA#1  W/ 
Control  LEA  #1 


Experiment  LEA#2  W/ 
Control  LEA  #1 


Experiment  LEA#3  W/ 
Control  LEA  #2 


Experiment  LEA#4  W/ 
Control  LEA  #2 


N 

28 


23 
33 

2J 
127 


X 

5.250 
5.363 


Tl 


1 

1.174 
1.112 


§  T2 


Spring  '71  T 


N 

25 

17 
23 


91 


5.762 


2.681 


1.537 


1.916 


Signif . 


,05 


NS 


NS 


NS 


X 

4.920 

4.705 
4.652 


T2|T3 


SD 
1.552 

0.985 
1.721 


25.502 


5.735 


3.322 


-Q.405 


Signif 


,001 


NS 


NS 


Col 


Row 


Col 


Row 


Col 


Row 
TfoT 


Row 


Fall  '71  T 


TT 

32 
21 


24 

.ID 
36 

82 


 T" 

6.031 


MB 


5.J^ 
5.083 


T3  §  T4 


3 

0.93 


1  .4?f 
1.40  ' 


0.799 


5..A48 


0.448 


1.476 


0.018 


8.202 


0.150 


.0.689 


0.278 


0.849 


0.26S 


0.145 


Signif. 


NS 


J2S_ 


NS 


uQl. 


JB  


jaa. 


NS 


Spring  '72 


TT 

31 


-22 
Z2 


-27 
-2D 
33 


5.80d 


5.JL2: 
5.4'if 


5.00C 


SD 
1.0 


n  ,  9':8 


_L^;.o 

IL-9'-4 


T1ST4 


5.189 


1.73.1 


n.nnn 


Signif. 


J15_ 


0 

8. 

6 
4 
2 
0 
8 
6 
4 
2 
0 
8 
6 
4 
2 
0 
8 
6 
4 
2 
0 
8 
6 
4 
2 
0 
8 
6 
4 
2 
0 
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QUBSTIONNAIRE  DATA 


As  I  see  it.  mv  oracmization  has  made  progress  in 
attaining  its  objectives. 


SCALE 
VALUES 


7.0 
6.8 
6.6 
6.4 
6.2 
6.0 
5.8 
5.6 
5.4 
5.2 
5.0 
4.8 
4.6 
4.4 
4.2 
4.0 
3.8 
3.6 
3.4 
3.2 
3.0 
2.8 
2.6 
2.4 
2.2 
2.0 
1.8 
1.6 
1.4 
1.2 
1.0 


Fall,  1971 


T3 

6. 

X 

125 

5. 

483 

5. 

750 

5. 

241 

5. 

733 

5. 

305 

Spring,  1972 


*  •  ' 


Experimental  LEA  #1 
Experimental  LEA  #2 
Experimental  LEA  #3 
Experimental  LEA  #4 
Control  LEA  #1 
Control  LEA  #2 

Total  


N 

22 
_31 

M 

29 

36 
182 


SD 
0.94 

1.09 


0.79; 


0.95( 


N 

n 

32 
23 
27 


0.94^ 


I.IK 


33 


166 


X 

5.9o: 


5.37E 


5.478 


5.600 
5.303 


SD 
0.78} 


1.07 


0.94' 


4.81^  1.21( 


1.07^ 


Two-Wav  Analysis  of  Var. 

T3    8  T4 

Experiment  LEA#1 
W/Exneriment  LEA#2 

Col. 

0.894 

NS 

Row 

.001 

Experiment  LEA#3 
W/Experiment  LEA#4 

Col. 

3.156 

NS 

Row 

8.892 

.01 

Experiment  LEA#1 
J|l/Control  LEA#1 

Col. 

1.129 

NS 

Row 

4,?24 

.05 

Experiment  LEA#2 
^/^^.or^tra\  LEA#1 

CpI.  , 

0.403 

NS 

Row 

1.548 

NS 

Experiment  LEA#3 
W/Control  LEA#2 

Col. 

0.512 

NS 

Row 

2.614 

■    !TS""  ■ 

Experiment  LEA#4 
W/Control  LEA#2 

Col. 

1.193 

NS 

Row 

1.978 

NS 
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APPENDIX  D 
QUESTIONNAIRE 


590 


Directions 


1.  Read  each  statement  carefully. 

2.  Then  deci(?G  to  what  extent  the  statement  accurately 

describes  your  work  situation  as  it  was  last  fall 
(September-December) .    Choose  a  numheFvalue  on — 
the  scale  which  best  shows  how  you  felt  th«n; 
write  your  choice  on  the  left  side  of  the~nage. 

3 .  Next  decide  to  what  extent  this  same  statement 

accurately  describes  your  work""iItuation  as  it  is 
now.    Choose  the  number  on  the  scale  whicFTbest 
shows  how  you  feel  this  day  and  mark  it  on  the 
right  side  of  the  page. 

4.  Be  sure  to  indicate  your  feeling  on  all  statements 

in  both  the  past  and  Present  tense. 


EX/JIPLE 


Now 


not  at  all         fairly  often  very  often 

Last  Fall 

  Our  organizational  plan  is  well  (44) 

formulated.   

LAST  PALL 


NOW 


1.  As  I  see  it,  persons  in  this  organi-  (58) 
zation  nut  a  lot  of  effort  into 
planning. 

2.  My  manager  knows  and  understands  (59) 
the  problems  I  face. 

3.  My  manager  recognizes  when  a  problem  (60) 
is  developing  and  does  something 
constructive  about  it. 

4.  My  work  is  important  to  the  future  (61) 
and  quality  of  education  in  my  state. 


59^ 


12         3         4         5         6  7 
not  at  all         fairly  often  very  often 

LAST  FALL 

  5,    When  differences  arise  in  my  work  (62) 

group^  we  have  good  ways  for 
settling  them  ourselves • 

  6.    I  think  that  my  work  grouD  has  made  (63) 

progress  in  attaining  the  objectives 
set  for  us, 

  7.    I  think  that  the  right  strategies  (64) 

are  chosen  to  implement  the 
objectives  of  my  organization. 

  8,    The  planning  unit  has  been  helpful  (65) 

to  me, 

  9,    My  organization's  overall  plan  is  (66) 

operable, 

  10,    In  this  organization  good  job  (67) 

performance  is  appropriately 
rewarded. 

  11.    I  feel  loyal  to  my  organization  and  (68) 

identify  with  it  and  its  objectives. 

  12.    My  capability  to  plan  effectively  (69) 

will  positively  affect  my  future 
career  in  this  organization. 

  13.    People  here  are  open  and  honest  in  (70) 

talking  with  each  other. 

  14.    I  am  given  appropriate  oppor-(Cdl.  2)  (01) 

tunitics  to  gain  more  techni- 
cal knowledge  about  my  job. 

  15.    My  organization  has  reliable  ways  (02) 

for  knowing  how  well  it  is  attain- 
ing its  objectives. 

  16.    My  manager  understands  planning  (03) 

tueory  and  is  able  to  put  it  into 
practice. 

  17.    The  policy  and  objectives  of  this  (04) 

organization  are  under stoorl  and 
supported  by  its  members. 
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12         3         4         5         6  7 
not  at  all         fairly  often  very  often 

LAST  FALL  NOW 

  18.     I  have  good  ways  for  knowing  how  (05)  

qood  our  results  are. 

  19.    My  organization's  performance  (06)  

standards  which  are  used  to 
evaluate  how  well  we  are  achieving 
our  objectives  are  understood. 

  20.     I  believe  that  all  persons  or  (07)  

groups  who  should  have  partici- 
pated iji  the  development  of 
objectives  did  participate. 

  21.    My  manager  encourages  and  supports  (08)  

innovation. 

  22.    I  feel  my  group  works  together  well. (09)  

  23.    I  am  able  to  affect  decisions  (10)  

relating  to  planning. 

  24.    The  top  priority  objectives  for  (11)  

state  education  are  clear  to  me. 

  25.    Based  on  information  I  have  (12)  

received  from  my  manager,  I  know 
if  I  an  measuring  up  in  my  job. 

  26.    I  can  influence  the  goals,  methods,  (13)  

and  activities  of  my  organization. 

  27.     I  understand  what  results  must  be  (14)  

produced  to  achieve  the  stated 
objectives  of  this  organization. 

  28.     The  kinds  of  things  I  am  doing  (15)  

will  make  a  long  term  contribution 
to  education. 

  29.     I  think  that  neople  in  this  organi-  (16)  

zation  are  generally  favorable 
toward  it  and  support  behavior 
implementing  its  objectives. 

  30.    ily  manager  shows  confi(?.cncc  and  (17)  

trust  in  me. 

  31.    I  really  feel  my  immediate  work  (18)  

group  is  clotting  things  done. 


ERLC 
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not 

LZ\ST  FALL 
  32. 

  33. 

  34. 

  35. 

  36. 

  37* 

  38. 

  39. 

  40. 

  41. 

  42. 

  43. 

  44. 

  45. 

 ^  46. 

ERiC 


12         3         4         5         6  7 
at  all  fairly  oftc^n  very  often 

NOTT 

As  I  see  it^  planning  is  an  integral  (19)  

part  of  running  the  state *c  schools. 

Our  goals  are  realistic  and  attain-  (20)  

able  with  our  best  efforts « 

I  feel  that  the  appropriate  persons  (21)  

and  groups  have  been  involved  in 
the  development  of  objectives  for 
this  organization. 

I  find  my  work  personally  rewarding  (22)  

and  meaningful. 

I  believe  my  organization  gives  me  (23)  

adequate  training  to  do  my  work 
effectively. 

The  work  I  do  makes  good  use  of  my  (24)  

abilities « 

People  in  my  group  have  the  technical (25)  

knowledge  to  do  the  job. 

As  I  see  It^  my  organization  has  (26)  

made  progress  in  attaining  its 
objectives. 

I  am  appropriately  involved  in  (27)  

decisions  affecting  my  work. 

Higher  management's  reactions  to  (28)  

the  problems  which  reach  them  are 
fair. 

The  activities  relating  to  planning  (29)  

are  having  an  effect  on  the  policy 
of  this  organization. 

I  and  my  manager  work  well  together.  (30)  

My  work  group  unc'erstanc's  what  we  (31)  

are  trying  to  achieve. 

I  feel  good  about  my  manager's  (32)  

ability  to  plan. 

My  group  works  hard  to  achieve  its  (33)  

goals. 
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1         2         3  5         6  7 

LAST  PALL 

  '"'7.    My  manager  makes  it  clear  that  he  is  (34) 

committed  to  the  success  of  our 
projects. 

  48.    As  I  see  it,  my  crcanization  is  (35) 

moving  in  the  right  direction. 

  49.    The  goals  of  this  organxzation  are  (36) 

articulated. 

  50.    Good  ways  are  used  to  let  me  know  (37) 

how  I  can  improve  my  performance. 

  51.    The  people  I  work  with  participate  (38) 

appropriately  in  setting  the  goals 
of  our  work. 

  52.    My  organization's  policy  statements  (39) 

are  clear. 

  53.    My  manager  provides  ne  with  adequate  (40) 

support  to  perform  my  job. 

  54.    My  manager  has  expressed  the  belief  (41) 

that  the  AMA's  training  program  has 
been  helpful. 

  55.     I  think  that  the  objectives  (42) 

developed  during  ATiA  training  are 
clearly  stated  with  rogaird  to 
results  expected.  ' 

  56.    I  feel  that  the  objectives  (43) 

developed  durinq  7\fiA  training 
reflect  the  most  serious  and  press- 
ing needs  of  state  education.^ 

  57.    As  I  sec  it,  the  operational  (44) 

priorities  of  the  objectives 
developed  during  ?J1A  training  arc 
clear. 
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APPENDIX  E 
INTERVIEW  CONTENT  CODING  INSTRUIIENT 


6U4 


Question  ffl 


V^hat  clo  you  feel  you  (will  obtain)  (obtained) 
from  the  AilA  training? 

pre-training  post-training 


Domains 

1.  definition  of  the 
institution's  mission 

2.  modify  previously  esteib- 
lished  objectives 

3.  identify  and  analyze 
alternative  courses  of 
action 

4.  determine  priorities 

5.  define  standards  of 
performance  for  key 
administrators 

6.  specify  task  completion 
dates  and  action 
assignments. 

7.  assign  responsibilities 
to  subordinate  units 

8.  design  a  methodology  by 
v;hich  future  performance 
may  be  evaluated  in  rela- 
tion to  the  performances 
specified  in  the  plan 

9.  produce  anr»  implement  a 
long-range  strategic  plan 

10.  establish  crodibility  of 
planning 

11.  promote  froc  flo«;  of  infor- 
mation throughout  SED 

12.  promote  cooperative  team 
work 


Intensity  Scale 

0  uncodeablo 
(no  cinswer) 

1  no  value 
2 

little  value 


4  include*-!  as 
circumstances 
permit 


significant 
should  be 
stressed 


7  maximiun  commit- 
ment 


Column 
Number  Score 


3 
4 


6 
7 


8 
9 
10 
11 
12 
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QuGstion  #2 


Do  you  foel  that  thc^  objectives  developed  as  a  result 
of  AMA  training  reflect  the  most  serious  and  pressing 
needs  of  state  education? 


Domain 


Question  acta  as  donain 
in  this  case. 


13. 


Intensity  Scale 

0  uncodeable 
(no  response) 


not  at  all 


to  some  extent 


definitely 


13 
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Question  #3 

How  are  ma jo«^ decisions  made  in  the  Ptate  Department? 
nonains 

Question  acts  as  domain  in  this  case. 

Intensity  Scales 


Column 
Number  Score 


34.  Involvement 

35.  0  uncodeable  (no 

response) 

1  no  participation/no 
discussion  invited, 
necision-m^iking  only 
at  top. 


no  participation/some 
discussion  invited. 
Decision-making  at  top. 


4  some  participation  as 
circumstances  permit. 
Decision-making 
mostly  at  ton. 


[B]  Quality 

0 


uncoaeable 
(no  response) 

never  effective 
seldom  effective 


sometimes 
effective 


usually  effective 
highly  effective 


34 
35 


significant  partici- 
pationAroad  policy 
at  top 


7  maximum  participation 
throughout  ST^ 
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Question  #4 


ERIC 


l^at  Is  the  role 
school  system? 


of  planning  in  running  the  State's 


Domain 

Questioi*  acts  as  domain 


*52   [A]  Role  of 
Planning 

0  uncodeable  (no 
answer) 


♦53  IBI  Need  for 
Planning 

0  uncodeable  (no 
answer) 


no  value 


little  value 


used  as  circum- 
stances permit 


significant 


integral  part 


no  value—should  not 
be  used  at  all 


little  value — should 
ho  used  less 


used  about  as  much 
as  it  should  be 


significant — should 
be  used  more 


everything  should  be 
thoroughly  planned 


*54  [C]  Emergence  of  Planning 
0  uncodeable  (no  answer) 


still  not  used 


recent  development 
long-standing  practice 


Column 
Number  Score 

A) 


B) 


C) 
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Question  #5 


Is  planning  influencing  the  decision-making  process 
v/ithin  the  State  Department  of  Education? 

Question  acts  as  domain  in  this  case. 

Intensity  Scale 


16. 


0  uncodcable  (no 
response) 


Column 
Number  Score 

16 


no  influence 
little  influence 
some  influence 
significant  influence 


integral  part  of 
decision-making  process 


ERIC 
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Question  #6 

How  has  the  planning  unit  helped  you  to  plan? 


Domain 

17.  awareness  of  need  to 
evaluate  our  programs 

18.  available  to  answer 
planning  questions 

19.  writing  guidelines  for 
plan  development 

20.  reviewing  and  refining 
plans 

21.  provides  leadership  in 
the  implementation  of 
planning 

22.  provides  in-sarvice 
training  in  planning 


Intensity  Scale 

0  uncodeable  (no 
response) 


1 
2 
3 
4 
5 
6 


no  help 


some  help 


great  help 


17. 
18. 
19. 
20. 

21. 
22. 


ERIC 
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Question  #7 


Toward  what  action  plan  objectives  has  measurable 
progress  been  made  by  your  division? 

Domains 

Question  acts  as  domain  in  this  case. 


23. 

[A)  Number  of  objec- 
tives toward 
which  progress 
has  been  made. 


24. 

(BJ  Level  of  pro- 
gress toward 
those  objec- 
tives. 


25. 

[C]  Time  as  a 
factor  in 
level  of 
progress 


0  uncodeable  (no 
response) 


0  uncodeable  (no     0  uncodeable  (no 
response)  response) 


1 

2 

3 
4 
5 
6 
7 


no  objectives 


some  objectives 


many  objectives 


no  progress 


some  progress 


much  progress 


1 
2 

3 
4 
5 
6 
7 


no  importance 


some  importance 


very  important 


23 


24 


25 
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Question  $8 


How  do  you  feel  about  the  direction  your  organiza- 
tion is  moving? 


Domain 

51.    Question  acts  as 

domain  in  this  case. 


Intensity  Scale 

0  uncodeable  (no 
answer) 


not  satisfied 
at  all 


slightly 
satisfied 


mostly  satisfied 


ccmpletely 
satisfied 


Column 
Number  Score 


51. 


7 
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Question  #9 


mat  changes  in  tho  planning  process  or  in  tho 
action  plan  itself  do  you  feel  necessary  at  this 
time? 


Domains 

26.  Increased  involvement 
of  professional  staff 
with  planning 

27.  Adjustment  of  org. 
structure  and  functions 
to  fit  plan 

28.  More  time  away  from 
routine  duties  for 
planning 

29.  More  efficient  use  of 
man  power  in  division- 
level  planning 

30.  Less  lead  time  between 
planning  and  implemen- 
tation 

31.  Less  talk  about  planning 
and  more  actut 1  planning 

^2.    Communication  and 
coordination 

33.    Greater  involvement 
in  lea's 


Intensity  Scale 

0  uncodeable 
(no  response/ 


1 
2 

3 
4 

5 
6 
7 


maximum  change 


significant 
change 


moderate 
chanae 


slight  change 
no  change 


26, 

27  o 

28. 

29. 

30. 
31. 
32. 
33. 


Action  Plan 

55^    Priorities  established 
among  ob j ect ives 

56.  Cost  estimates  attached 
to  objectives 

57.  Addition  of  valuation 
strategies 

58.  Operational  definitions 
of  terms  used  in 
objectives 

59.  Assigned  responsibili- 
ties for  implementation 

60.  General  revision  of  plans 


55. 
56. 
57. 

58. 

59. 
60. 
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QuGStlon  #10 


W^at  are  the  roadblocks  to  change  in  this 
organization? 


Domains 

36.  organization  reacts  to 
problems  rather  than 
anticipates  and  deals 
v;ith  problems 

37.  public  relations 
(role  of  Pressure 
groups ) 

38.  adequate  resources 
(money  and  informa- 
tion) 

39.  control  system  ex- 
pressed through 
decision^-maUcing 
process 

40.  sense  of  SED  mission 

41.  employee  interper- 
sonal skills 

42.  2iinount  of  cooperative 
teamwork  present 

43.  infoiroal  power  group- 
ings supporting  or 
opposing  goals  of 
formal  organization 

44.  degree  to  which  per- 
sons within  organiza- 
tion will  support 
change 


Intensity  Scale 

0  uncodeable  (no 
answer) 


major  roadblock/ 
always  stops 
change 


significant  ^ 
roadblock/usually 
stops  change 


occasional 
roadblock 


i^eak  roadblock/ 
seldom  stops 
desired  change 


Column 
Numbr.r  Score 


36. 


37. 


38. 


39. 
40. 

41. 

42. 


43. 


44. 
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12.    What  specific  methods  do  you  use  to  detormine  if 
the  continuing  and  specific  objectives  of  your 
division  are  being  met? 


Domains 


5. 
6. 
7. 
8. 

9. 


4.    client  satisfaction 

Purveys  (opinion  polls t 
review  of  media  com- 
mentaries) 

custom  made  tests 

independent  evaluators 

informal  feedback 

performance  reviews 
(interviews^  meetings) 

product  analysis 
(inspecting  and  rating 
of  products  which 
students  have  made^ 
e.g. I  handwriting f 
drawings,  projects  or 
which  administrators 
have  developed,  e.g., 
plans ^  job  descriptions, 
reports) 

10.  questionnaires 

11.  standardized  tests 

12.  task  completion  inven-* 
toriGS  (money  appropri- 
ated,  reports  completed, 
training  sessions  given, 
plans  turned  in) 

13.  unobtrusive  measures 
(observations,  organi- 
zational  documents) 

14.  written  progress  reports 

15.  index  combining  multiple 
indices  (production 
functions) 


0  uncodeable 


minimum  use 


moderate  use 


meucimum  use 


4 
5 
6 
7 

8 


9 
10 
11 


12 

13 
14 

15 
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11.  Have  rorformance  standards  been  established  for  your 
subordinates  based  on  the  objectives  in  your  division 
plan? 

12.  Do  you  have  regular  performance  reviews  with  your 
subordinates? 


Question  acts  as  domain 


Column 
NuRiber  Score 


(1)  Extent  of  Use 


0  tmcodeable  (no 
response) 


none  at  all 


to  some  extent 


definitely 


(2)  Need  for 
Performance 
Standards 

0  uncodeable  (no 
response) 


no  value— should 
not  be  used  at  all 


little  value- 
should  be  used 
less 


should  be  used  more 


should  be  used 
much  more 


1) 
2) 
3) 


(3)  Existence  of  Performance 
Reviews 


ERJC 


0  uncodeable 
1 


still  not  used 


used  to  some  extent 


regular  performance 
reviews  held 


